Re: Python while loop
In <0c642381-4dd2-48c5-bb22-b38f2d5b2...@googlegroups.com> paul.garcia2...@gmail.com writes: > Write a program which prints the sum of numbers from 1 to 101 > (1 and 101 are included) that are divisible by 5 (Use while loop) > x=0 > count=0 > while x<=100: > if x%5==0: > count=count+x > x=x+1 > print(count) > > Question: How does python know what count means? "count" is an english word meaning "how many things do I have?", but python doesn't know that. In python, "count" is just a name; you could have called it "hamburger" and python would treat it just the same. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, "The Gashlycrumb Tinies" -- https://mail.python.org/mailman/listinfo/python-list
Re: Python while loop
On 29/11/2016 23:58, paul.garcia2...@gmail.com wrote: Write a program which prints the sum of numbers from 1 to 101 ( 1 and 101 are included) that are divisible by 5 (Use while loop) This is the code: x=0 count=0 while x<=100: if x%5==0: count=count+x x=x+1 print(count) This looks at numbers from 0 to 100 inclusive, not 1 to 101. Although it doesn't affect the result. (It will add in 0 which is divisible by 5, but that doesn't change it either. If this is an exercise however, checking the correct range might be important!) -- Bartc -- https://mail.python.org/mailman/listinfo/python-list
Re: Python while loop
On 2016-11-29 23:58, paul.garcia2...@gmail.com wrote: Write a program which prints the sum of numbers from 1 to 101 ( 1 and 101 are included) that are divisible by 5 (Use while loop) This is the code: x=0 count=0 while x<=100: if x%5==0: count=count+x x=x+1 print(count) Question: How does python know what count means ? I see that its declared. What i wanna know is how does it know the iteration or each number it finds divisible by 5 is the "Count" ?? It doesn't, it's just a name, and, anyway, in the code, 'x' is the count... If it _did_ know (which is doesn't), wouldn't it be complaining that 'count' isn't the count but the sum? :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
On Monday, 30 June 2014 18:16:21 UTC+5:30, Peter Otten wrote: Jaydeep Patil wrote: I have did excel automation using python. In my code I am creating python dictionaries for different three columns data at a time.There are are many rows above 4000. Lets have look in below function. Why it is taking too much time? Code: def transientTestDict(self,ws,startrow,startcol): self.hwaDict = OrderedDict() self.yawRateDict = OrderedDict() rng = ws.Cells(startrow,startcol) while not rng.Value is None: r = rng.Row c = rng.Column time = rng.Value rng1 = rng.GetOffset(0,1) hwa = rng1.Value rng2 = rng.GetOffset(0,2) yawrate = rng2.Value self.hwaDict[time] = hwa,rng.Row,rng.Column self.yawRateDict[time] = yawrate,rng.Row,rng.Column rng = ws.Cells(r+1,c) Please have look in above code suggest me to improve speed of my code. Assuming that what slows down things is neither Python nor Excel, but the communication between these I'd try to do as much as possible in Python. For example (untested): def transientTestDict(self, ws, startrow, startcol): self.hwaDict = OrderedDict() self.yawRateDict = OrderedDict() time_col, hwa_col, yawrate_col = range(startcol, startcol+3) for row in xrange(startrow, sys.maxint): time = ws.Cells(row, time_col).Value if time is None: break hwa = ws.Cells(row, hwa_col).Value yawrate = ws.Cells(row, yawrate_col).Value self.hwaDict[time] = hwa, row, time_col self.yawRateDict[time] = yawrate, row, time_col While this avoids cell arithmetic in Excel it still fetches every value separately, so I have no idea if there is a significant effect. Does Excel provide a means to get multiple cell values at once? That would likely help. Dear Peter, I have tested code written by you. But still it is taking same time. Regards Jay -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
Jaydeep Patil wrote: Dear Peter, I have tested code written by you. But still it is taking same time. Too bad ;( If you run the equivalent loop written in Basic from within Excel -- is that faster? If you run the loop in Python with some made-up data instead of that fetched from Excel -- is that faster? What I'm trying to tell you: you need to put in some work to identify the culprit... -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
On Tue, 01 Jul 2014 14:40:18 +0200, Peter Otten wrote: What I'm trying to tell you: you need to put in some work to identify the culprit... His next question was how do I read a range from excel, please give me an example I gave him an example of using google to search for solutions to his problem. If he can't be bothered to try and solve it himslef, I'm nopt going to write his code for him. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
Jaydeep Patil wrote: I have did excel automation using python. In my code I am creating python dictionaries for different three columns data at a time.There are are many rows above 4000. Lets have look in below function. Why it is taking too much time? Code: def transientTestDict(self,ws,startrow,startcol): self.hwaDict = OrderedDict() self.yawRateDict = OrderedDict() rng = ws.Cells(startrow,startcol) while not rng.Value is None: r = rng.Row c = rng.Column time = rng.Value rng1 = rng.GetOffset(0,1) hwa = rng1.Value rng2 = rng.GetOffset(0,2) yawrate = rng2.Value self.hwaDict[time] = hwa,rng.Row,rng.Column self.yawRateDict[time] = yawrate,rng.Row,rng.Column rng = ws.Cells(r+1,c) Please have look in above code suggest me to improve speed of my code. Assuming that what slows down things is neither Python nor Excel, but the communication between these I'd try to do as much as possible in Python. For example (untested): def transientTestDict(self, ws, startrow, startcol): self.hwaDict = OrderedDict() self.yawRateDict = OrderedDict() time_col, hwa_col, yawrate_col = range(startcol, startcol+3) for row in xrange(startrow, sys.maxint): time = ws.Cells(row, time_col).Value if time is None: break hwa = ws.Cells(row, hwa_col).Value yawrate = ws.Cells(row, yawrate_col).Value self.hwaDict[time] = hwa, row, time_col self.yawRateDict[time] = yawrate, row, time_col While this avoids cell arithmetic in Excel it still fetches every value separately, so I have no idea if there is a significant effect. Does Excel provide a means to get multiple cell values at once? That would likely help. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
On Monday, June 30, 2014 1:32:23 PM UTC+2, Jaydeep Patil wrote: I have did excel automation using python. In my code I am creating python dictionaries for different three columns data at a time.There are are many rows above 4000. Lets have look in below function. Why it is taking too much time? Code: def transientTestDict(self,ws,startrow,startcol): self.hwaDict = OrderedDict() self.yawRateDict = OrderedDict() rng = ws.Cells(startrow,startcol) while not rng.Value is None: r = rng.Row c = rng.Column time = rng.Value rng1 = rng.GetOffset(0,1) hwa = rng1.Value rng2 = rng.GetOffset(0,2) yawrate = rng2.Value self.hwaDict[time] = hwa,rng.Row,rng.Column self.yawRateDict[time] = yawrate,rng.Row,rng.Column rng = ws.Cells(r+1,c) Please have look in above code suggest me to improve speed of my code. Regards Jaydeep Patil Hi Jaydeep, I agree with Peter. I would avoid moving from cell to cell through the EXCEL interface if you can avoid. If possible, I would try to read ranges from EXCEL into a python list (or maybe numpy arrays) and do the processing in Python. In the past I even dumped an EXCEL sheet as a CSV file and then used the numpy recfromcsv function to process the data. If you are really brave, dump EXCEL alltogether :) and do all the work in Python (have you already tried IPython notebook?). Regards, Marco -- https://mail.python.org/mailman/listinfo/python-list
Re: Python While loop Takes too much time.
marco.naw...@colosso.nl wrote: In the past I even dumped an EXCEL sheet as a CSV file That's probably the only way you'll speed things up significantly. In my experience, accessing Excel via COM is abysmally slow no matter how you go about it. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: python for loop
The first item in a sequence is at index zero because it is that far away from the beginning. The second item is one away from the beginning. That is the reason for zero-based indexing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
Hi Roy,
On Feb 11, 2013, at 10:24 AM, Roy Smith r...@panix.com wrote:
Is this server that you're talking to something that you have control
over, i.e. are you stuck with this protocol? Given a choice, I'd go
with something like JSON, for which pre-existing libraries for every
language under the sun.
I'm running JSON for my application messaging protocol but with JSON and python
default unordered dict,
there's no guarantee if I put in the length key in the JSON message, it will be
placed on the first bytes hence
why it was designed for a fixed 4-byte at the start of the message to indicate
the message length.
Beyond the 4-bytes it is all JSON.
but if you have a better idea, i would certainly welcome it.
Do you actually *know* what the value of nbuf is? Is it possible that
(somehow) it's 0? You should print (log, whatever), the value of nbuf,
just to make sure.
nbuf is printing the right bytes amount, I removed the print statement before I
made the first post.
So to clarify, I added a print statement between the first recv and the second.
{msgver: 1.0, msgid: 200, subcode: 100, appver: 1.0, appid:
1.0, data: {1: igb0, 2: igb1, ifcnt: 2}}
connected to misty:8080
sending data
138 bytes sent: 0x86{msgver: 1.0, msgid: 200, subcode: 100,
appver: 1.0, appid: 1.0, data: {1: igb0, 2: igb1, ifcnt:
2}}
receiving data
message length is 188
0 bytes received:
So the subsequent recv() call will be readjusted with 188 bytes buffer size so
theoretically, recv shouldn't return 0.
The same logic that I used to send to the server from the python client that
the server will readjust the second recv() call based on the length
information. On this 2nd recv() call the server is able to obtain the rest of
the messages.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
Hi Dave,
On Feb 11, 2013, at 9:22 AM, Dave Angel da...@davea.name wrote:
Exactly how are you sending hexadecimal ? If that 0xad (which is only one
byte, what about the other 3 ?) is intended to be a C description, then it's
certainly not hex, it's binary. And probably little-endian, to boot. That's
a mistake, as network protocols almost always use big-endian.
So what is the range of values for the length, and how are they actually
encoded? Are they uint32 in native binary format? And are you permitted to
change the encoding, to fix it? (If it were my choice, I'd make it a
printable decimal value, first choice, or printable hex, second choice.)
They are ASCII stream, actually JSON with the exception of the first 4 bytes. I
avoided using struct class for this as it's overkill for my application.
So the idea is that i code the length to a max of 0xff (max length of 256
bytes), I would only have to assign 4 bytes to it. If i need 1k length, i just
need to increase it to 6 bytes or 4 if i decide to strip the 0x.
the code for the function is as follows:
def request_get_session(sock, jmsg):
# append message length
plen = hex(len(jmsg))
msg = '{0}{1}'.format(plen, jmsg)
print 'sending data'
n = sock.send(msg)
str = '{0} bytes sent: {1}'.format(n, msg)
print str
# receive message length
print 'receiving data'
mlen = sock.recv(4)
try:
nbuf = int(mlen, 16)
except ValueError as e:
print 'invalid length type'
return -1
print 'message length is {0}'.format(nbuf)
while True:
buf = sock.recv(nbuf)
if not buf:
break
slen = len(buf)
str = {0} bytes received: {1}.format(slen, buf)
print str
return 0
I've managed to receive and translate the message length until I reach my
second recv which I readjusted the buffer size to include the new message
length.
However that failed and recv received 0 bytes. I implemented the same
algorithm on the server side using C and it work so appreciate if you can
help me on this.
# receive message length
print 'receiving data'
mlen = sock.recv(4)
try:
nbuf = int(mlen, 16)
That supposes the count is being sent as a printable string of hex digits.
That's not what I concluded above.
That is what I meant. it just an ascii string.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
Hi MRAB,
My code now works thanks to your advice.
{msgver: 1.0, msgid: 200, subcode: 100, appver: 1.0, appid:
1.0, data: {1: igb0, 2: igb1, ifcnt: 2}}
connected to misty:8080
sending data
138 bytes sent: 0x86{msgver: 1.0, msgid: 200, subcode: 100,
appver: 1.0, appid: 1.0, data: {1: igb0, 2: igb1, ifcnt:
2}}
receiving data
message length is 188
188 bytes received: { msgver: 1.00, appver: 1.00, appid: 1000,
msgid: 10, subcode: 20, data: [ 110.159.183.16, 124.108.16.94,
2400:3700:50::2, 2400:3700:50::1, 2400:3700:51:: ] }
Many thanks.
On Feb 11, 2013, at 9:55 AM, MRAB pyt...@mrabarnett.plus.com wrote:
You should keep reading until you get all need or the connection is
closed:
buf = b''
while len(buf) nbuf:
chunk = sock.recv(nbuf - len(buf))
if not chunk:
break
buf += chunk
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On 2013-02-11 14:56, Ihsan Junaidi Ibrahim wrote:
Hi Roy,
On Feb 11, 2013, at 10:24 AM, Roy Smith r...@panix.com wrote:
Is this server that you're talking to something that you have control
over, i.e. are you stuck with this protocol? Given a choice, I'd go
with something like JSON, for which pre-existing libraries for every
language under the sun.
I'm running JSON for my application messaging protocol but with JSON and python
default unordered dict,
there's no guarantee if I put in the length key in the JSON message, it will be
placed on the first bytes hence
why it was designed for a fixed 4-byte at the start of the message to indicate
the message length.
Beyond the 4-bytes it is all JSON.
but if you have a better idea, i would certainly welcome it.
I probably wouldn't make it fixed length. I'd have the length in
decimal followed by, say, \n.
Do you actually *know* what the value of nbuf is? Is it possible that
(somehow) it's 0? You should print (log, whatever), the value of nbuf,
just to make sure.
nbuf is printing the right bytes amount, I removed the print statement before I
made the first post.
So to clarify, I added a print statement between the first recv and the second.
{msgver: 1.0, msgid: 200, subcode: 100, appver: 1.0, appid: 1.0, data: {1: igb0,
2: igb1, ifcnt: 2}}
connected to misty:8080
sending data
138 bytes sent: 0x86{msgver: 1.0, msgid: 200, subcode: 100, appver: 1.0, appid: 1.0, data: {1:
igb0, 2: igb1, ifcnt: 2}}
receiving data
message length is 188
0 bytes received:
So the subsequent recv() call will be readjusted with 188 bytes buffer size so
theoretically, recv shouldn't return 0.
The same logic that I used to send to the server from the python client that
the server will readjust the second recv() call based on the length
information. On this 2nd recv() call the server is able to obtain the rest of
the messages.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On Tue, Feb 12, 2013 at 2:11 AM, MRAB pyt...@mrabarnett.plus.com wrote: I probably wouldn't make it fixed length. I'd have the length in decimal followed by, say, \n. Or even followed by any non-digit. Chances are your JSON data begins with a non-digit, so you'd just have to insert a space in the event that you're JSON-encoding a flat integer. (Which might not ever happen, if you know that your data will always be an object.) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On 02/11/2013 10:02 AM, Ihsan Junaidi Ibrahim wrote:
snip
print 'message length is {0}'.format(nbuf)
while True:
buf = sock.recv(nbuf)
if not buf:
break
This loop doesn't terminate till buf is zero length, which it will be
eventually. At that point, you've overwritten the real data you may
have gotten. So the loop is just plain wrong.
Uwe MRAB's code, since there's no promise that all the data will be
returned in a single call. Keep accumulating it as you loop.
slen = len(buf)
str = {0} bytes received: {1}.format(slen, buf)
print str
return 0
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On Feb 11, 2013, at 11:24 PM, Chris Angelico ros...@gmail.com wrote: On Tue, Feb 12, 2013 at 2:11 AM, MRAB pyt...@mrabarnett.plus.com wrote: I probably wouldn't make it fixed length. I'd have the length in decimal followed by, say, \n. Or even followed by any non-digit. Chances are your JSON data begins with a non-digit, so you'd just have to insert a space in the event that you're JSON-encoding a flat integer. (Which might not ever happen, if you know that your data will always be an object.) ChrisA So on the first recv() call, I set the buffer at 1 character and I iterate over single character until a non-digit character is encountered? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On Tue, Feb 12, 2013 at 12:41 PM, Ihsan Junaidi Ibrahim ih...@grep.my wrote: On Feb 11, 2013, at 11:24 PM, Chris Angelico ros...@gmail.com wrote: On Tue, Feb 12, 2013 at 2:11 AM, MRAB pyt...@mrabarnett.plus.com wrote: I probably wouldn't make it fixed length. I'd have the length in decimal followed by, say, \n. Or even followed by any non-digit. Chances are your JSON data begins with a non-digit, so you'd just have to insert a space in the event that you're JSON-encoding a flat integer. (Which might not ever happen, if you know that your data will always be an object.) ChrisA So on the first recv() call, I set the buffer at 1 character and I iterate over single character until a non-digit character is encountered? More efficient would be to guess that it'll be, say, 10 bytes, and then retain any excess for your JSON read loop. But you'd need to sort that out between the halves of your code. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
In article mailman.1655.1360594595.2939.python-l...@python.org, Ihsan Junaidi Ibrahim ih...@grep.my wrote: I'm running JSON for my application messaging protocol but with JSON and python default unordered dict, there's no guarantee if I put in the length key in the JSON message, it will be placed on the first bytes hence why it was designed for a fixed 4-byte at the start of the message to indicate the message length. Beyond the 4-bytes it is all JSON. I'm confused. It sounds like you're making things way more complicated than they have to be. Can you give us an example of an actual data message? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On 2013-02-12 02:20, Chris Angelico wrote: On Tue, Feb 12, 2013 at 12:41 PM, Ihsan Junaidi Ibrahim ih...@grep.my wrote: On Feb 11, 2013, at 11:24 PM, Chris Angelico ros...@gmail.com wrote: On Tue, Feb 12, 2013 at 2:11 AM, MRAB pyt...@mrabarnett.plus.com wrote: I probably wouldn't make it fixed length. I'd have the length in decimal followed by, say, \n. Or even followed by any non-digit. Chances are your JSON data begins with a non-digit, so you'd just have to insert a space in the event that you're JSON-encoding a flat integer. (Which might not ever happen, if you know that your data will always be an object.) ChrisA So on the first recv() call, I set the buffer at 1 character and I iterate over single character until a non-digit character is encountered? More efficient would be to guess that it'll be, say, 10 bytes, and then retain any excess for your JSON read loop. But you'd need to sort that out between the halves of your code. If the length is always followed by a space then it's easier to split it off the input: buf = sock.recv(10) space_pos = buf.find(b ) nbuf = int(buf[ : space_pos]) buf = buf[space_pos+ 1 : ] while len(buf) nbuf: chunk = sock.recv(nbuf - len(buf)) if not chunk: break buf += chunk I'm assuming that: 1. The initial recv returns the length followed by a space. It could, of course, return fewer bytes (space_pos == -1), so you may need to recv some more bytes, like what's done later on. 2. At least 10 bytes were sent. Imagine what would happen if the sender sent b2 [] immediately followed by b2 []. The initial recv could return all of it. In that case you could save the excess until next time. Alternatively, the sender could guarantee that it would never send fewer than the 10 bytes, padding with several b if necessary. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On 02/10/2013 07:48 PM, Ihsan Junaidi Ibrahim wrote:
Hi,
I'm implementing a python client connecting to a C-backend server and am
currently stuck to as to how to proceed with receiving variable-length byte
stream coming in from the server.
I have coded the first 4 bytes (in hexadecimal) of message coming in from the
server to specify the length of the message payload i.e. 0xad{...}
Exactly how are you sending hexadecimal ? If that 0xad (which is only
one byte, what about the other 3 ?) is intended to be a C description,
then it's certainly not hex, it's binary. And probably little-endian,
to boot. That's a mistake, as network protocols almost always use
big-endian.
So what is the range of values for the length, and how are they actually
encoded? Are they uint32 in native binary format? And are you
permitted to change the encoding, to fix it? (If it were my choice, I'd
make it a printable decimal value, first choice, or printable hex,
second choice.)
I've managed to receive and translate the message length until I reach my
second recv which I readjusted the buffer size to include the new message
length.
However that failed and recv received 0 bytes. I implemented the same algorithm
on the server side using C and it work so appreciate if you can help me on this.
# receive message length
print 'receiving data'
mlen = sock.recv(4)
try:
nbuf = int(mlen, 16)
That supposes the count is being sent as a printable string of hex
digits. That's not what I concluded above.
except ValueError as e:
If the count starts as 0xad, I can't imagine why this exception wasn't
thrown.
print 'invalid length type'
return -1
while True:
buf = sock.recv(nbuf)
if not buf:
break
slen = len(buf)
str = {0} bytes received: {1}.format(slen, buf)
print str
You might need to play with hexlify, if you persist in sending the count
in binary.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
On 2013-02-11 00:48, Ihsan Junaidi Ibrahim wrote:
Hi,
I'm implementing a python client connecting to a C-backend server and am
currently stuck to as to how to proceed with receiving variable-length byte
stream coming in from the server.
I have coded the first 4 bytes (in hexadecimal) of message coming in from the
server to specify the length of the message payload i.e. 0xad{...}
I've managed to receive and translate the message length until I reach my
second recv which I readjusted the buffer size to include the new message
length.
However that failed and recv received 0 bytes. I implemented the same algorithm
on the server side using C and it work so appreciate if you can help me on this.
# receive message length
print 'receiving data'
mlen = sock.recv(4)
try:
nbuf = int(mlen, 16)
except ValueError as e:
print 'invalid length type'
return -1
while True:
buf = sock.recv(nbuf)
if not buf:
break
slen = len(buf)
str = {0} bytes received: {1}.format(slen, buf)
print str
You should keep reading until you get all need or the connection is
closed:
buf = b''
while len(buf) nbuf:
chunk = sock.recv(nbuf - len(buf))
if not chunk:
break
buf += chunk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python recv loop
In article mailman.1612.1360544258.2939.python-l...@python.org,
Ihsan Junaidi Ibrahim ih...@grep.my wrote:
I'm implementing a python client connecting to a C-backend server and am
currently stuck to as to how to proceed with receiving variable-length byte
stream coming in from the server.
I have coded the first 4 bytes (in hexadecimal) of message coming in from the
server to specify the length of the message payload i.e. 0xad{...}
Is this server that you're talking to something that you have control
over, i.e. are you stuck with this protocol? Given a choice, I'd go
with something like JSON, for which pre-existing libraries for every
language under the sun.
But, let's assume for the moment that you're stuck with this
length-value encoding. OK, but it's going to be more complicated than
you think. [I assume we're talking TCP here?]
Carefully read the documentation for socket.recv():
socket.recv(bufsize[, flags]) [...] The maximum amount of data to be received
at once is specified by bufsize.
Linger on the word maximum, and try to grok the fullness of of how
annoying that can be. What it means is that if the other side sent 120
bytes (octets), recv() might return all 120 at once, or it might return
them one at a time, or anything in between.
So, what you need to do is call recv() repeatedly in a loop, each time
passing it a value for bufsize which represents the amount left in the
message (i.e. the original message length parsed earlier minus all the
little bits and pieces that have been read so far).
Keep in mind, you also need to do this when you recv() the first 4
octets, which make up the length field. What you've got, recv(4), will
work MOST of the time, but it's perfectly legal for recv() to return a
short read. You can't predict how fragmentation and retry timeouts and
all sorts of low-level crud will cause your message boundaries to get
scrambled.
# receive message length
print 'receiving data'
mlen = sock.recv(4)
try:
nbuf = int(mlen, 16)
except ValueError as e:
print 'invalid length type'
return -1
while True:
buf = sock.recv(nbuf)
if not buf:
break
slen = len(buf)
str = {0} bytes received: {1}.format(slen, buf)
print str
Do you actually *know* what the value of nbuf is? Is it possible that
(somehow) it's 0? You should print (log, whatever), the value of nbuf,
just to make sure.
And, once you'e got all this working, tear it all out and convert to
using something sane like JSON. Let somebody else worry about all the
horrible details.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 03:56 am, tjre...@udel.edu wrote: exar...@twistedmatrix.com wrote: There's a lot of things in Python that I don't strictly *need*. That doesn't mean that they wouldn't be welcome if I could have them. Getting rid of the range/xrange dichotomy would improve things. The developers agreed a couple of years ago. Starting using 3.1 if you want this. And there was much rejoicing, et cetera. Since 'range' could refer to a user-defined object, rather than the builtin function, there is no way the interpreter should substitute 'xrange'. See the earlier parts of this thread for the reasons this isn't true. :) Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 02:12 am, pavlovevide...@gmail.com wrote: On Aug 16, 3:35�pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like � �for i in range(n): as a simple integer loop. It would be a simple to do if you were writing it for a different langauge was a lot less dynamic than Python is. It'd be quite a complex hack to add it to CPython's compiler while maintaing the current highly dynamic runtime semantics and backwards compatibility, which is a design constraint of Python whether you like it or not. In your other message, you said this wasn't a legitimate CPython complaint. Here, you say that it would be a complex hack to implement this in CPython. complex hack has negative connotations in my mind. This seems contradictory to me. And all this complaining about an issue that can be worked around by xrange instead of range. Sheesh. Sure. The specific issue of range vs xrange is quite a small one. There is a slightly larger one regarding the flexibility (or relative lack thereof) of the CPython runtime, though. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 01:53 am, pavlovevide...@gmail.com wrote: On Aug 16, 6:28�pm, exar...@twistedmatrix.com wrote: On 01:23 am, benjamin.kap...@case.edu wrote: On Sun, Aug 16, 2009 at 6:35 PM, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like � for i in range(n): as a simple integer loop. In fact, Cython is able to do this. but special cases aren't special enough to break the rules. Although I think PyPy also recognizes this case and makes it as efficient as using xrange, and does so without breaking any rules. PyPy uses a JIT compiler (which is still slower than CPython, suggesting that perhaps they should have spent more time optimizing the general case than optimizing for an easily avoided special case). It's true that PyPy has a JIT compiler (which is still slower than CPython). However, this isn't where the range speedup comes from. The range optimization is handled specifically in the implementation of range (or possibly of list, or a combination of the two, I can't remember exactly). It's effective even when the JIT compiler is disabled. There *are* _some_ legitimate complaints to be made about the CPython runtime. :) This isn't one of them. xrange has been part of Python for 10 years. If there are any complaints to be made about this situation it's that there are any 2.x learning materials anythere that continue to use range() and not xrange() in this context. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 01:44 am, http wrote: exar...@twistedmatrix.com writes: Although I think PyPy also recognizes this case and makes it as efficient as using xrange, and does so without breaking any rules. How can pypy possibly know that the user hasn't assigned some other value to range? It doesn't really need to. The optimization isn't applied when the compiler sees the name range being called. It's applied after the object the default builtin name range is bound to is called. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 16 Aug, 19:12, Carl Banks pavlovevide...@gmail.com wrote: If you don't care about the dynamic stuff why don't you just use Cython? Or quit complaining and just use xrange. I think you are the only one complaining here. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
exar...@twistedmatrix.com wrote: On 02:12 am, pavlovevide...@gmail.com wrote: On Aug 16, 3:35�pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: � � � � Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like � �for i in range(n): as a simple integer loop. It would be a simple to do if you were writing it for a different langauge was a lot less dynamic than Python is. It'd be quite a complex hack to add it to CPython's compiler while maintaing the current highly dynamic runtime semantics and backwards compatibility, which is a design constraint of Python whether you like it or not. In your other message, you said this wasn't a legitimate CPython complaint. Here, you say that it would be a complex hack to implement this in CPython. complex hack has negative connotations in my mind. This seems contradictory to me. And all this complaining about an issue that can be worked around by xrange instead of range. Sheesh. Sure. The specific issue of range vs xrange is quite a small one. There is a slightly larger one regarding the flexibility (or relative lack thereof) of the CPython runtime, though. A possible solution would be to make 'range' return an instance of, say, 'RangeList', a subclass of list which would behave like 'list' externally but create the list of values internally only when needed. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Carl Banks wrote: On Aug 16, 3:35 pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. Cython can do this easily because it is a different language that is a lot less dynamic than Python. Actually, Cython is able to do this because it knows the global scope of a module. The only thing that this optimisation makes impossible when you compile your module using Cython is to inject builtins into the module namespace *after* the compilation, either by assigning module attributes or by importing the module into a custom namespace. Given that both use cases are extremely rare, it was decided that optimisations like this are more important than the ability to redefine the most common builtins (such as 'range' or 'enumerate'). So, in a way, Cython really makes them builtins. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, 16 Aug 2009 15:35:26 -0700, sturlamolden wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. Easily huh? Would you like to put a small wager on that? Here is an unedited copy-and-paste of the last few lines of an interactive Python session: for i in range(5): ... print i ... 0 1 2 Surprise! 4 __builtins__.range is range True Can you determine how I did this? How would the compiler avoid this? If you can find a method for the compiler to avoid surprises like this, why do you think it would be valuable to add all that extra complexity to the language? (As opposed to an external tool like Cython.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
John Machin wrote: On Aug 17, 8:35 am, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. Extremely easy, once users relinquish the right to replace built-in range with their own concoctions ... Cython allows you to do that. You just have to do it inside the module. If Cython determines that range refers the global builtin name, it will enable the loop optimisation. If you assign anything to that global name inside your module (even the range function itself), this will disable the optimisation. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, Aug 16, 2009 at 11:10 PM, Nobodynob...@nowhere.com wrote: Java also has iterators; it's more a case of people coming from C and BASIC. Although, some of those may have come *through* Java without abandoning old habits. You see the same thing with people coming from BASIC to C and writing: #define NUM_DATES 50 int day[NUM_DATES], month[NUM_DATES], year[NUM_DATES]; rather than defining a struct. Sometimes referred to as I know ten languages and can write in BASIC in all of them. Ha, ha. I learned that pattern in Fortran. I confess to having written code like that in C. I think I've gotten over it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Emmanuel Surleau wrote: Dr. Phillip M. Feldman wrote: [snip] def is_prime(n): for j in range(2,n): if (n % j) == 0: return False return True It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. [snip] I will also observe that if you were to stop programming whatever language you are more familiar with in Python, and start programming Python in Python, you'll have an easier time of it. I don't see what's particularly un-Pythonic with this code. Not using xrange() is a mistake, certainly, but it remains clear, easily understandable code which correctly demonstrates the naive algorithm for detecting whether n is a prime. It doesn't call for condescension [snip] Cheers, Emm My comment about programming Python in Python was geared more towards the subject line than the actual code, and the biases evident in his comments in both this thread and earlier ones. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 17, 4:40 am, exar...@twistedmatrix.com wrote: On 02:12 am, pavlovevide...@gmail.com wrote: On Aug 16, 3:35 pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. It would be a simple to do if you were writing it for a different langauge was a lot less dynamic than Python is. It'd be quite a complex hack to add it to CPython's compiler while maintaing the current highly dynamic runtime semantics and backwards compatibility, which is a design constraint of Python whether you like it or not. In your other message, you said this wasn't a legitimate CPython complaint. Here, you say that it would be a complex hack to implement this in CPython. complex hack has negative connotations in my mind. This seems contradictory to me. Well, you missed the point, chief. It's not a legitimate complaint because you can use xrange, you don't need compiler magic to recognize and optimize range. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 06:32 pm, pavlovevide...@gmail.com wrote: On Aug 17, 4:40�am, exar...@twistedmatrix.com wrote: On 02:12 am, pavlovevide...@gmail.com wrote: On Aug 16, 3:35�pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: � � � � Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like � �for i in range(n): as a simple integer loop. It would be a simple to do if you were writing it for a different langauge was a lot less dynamic than Python is. �It'd be quite a complex hack to add it to CPython's compiler while maintaing the current highly dynamic runtime semantics and backwards compatibility, which is a design constraint of Python whether you like it or not. In your other message, you said this wasn't a legitimate CPython complaint. Here, you say that it would be a complex hack to implement this in CPython. �complex hack has negative connotations in my mind. This seems contradictory to me. Well, you missed the point, chief. It's not a legitimate complaint because you can use xrange, you don't need compiler magic to recognize and optimize range. There's a lot of things in Python that I don't strictly *need*. That doesn't mean that they wouldn't be welcome if I could have them. Getting rid of the range/xrange dichotomy would improve things. Yes, I can work around it until the runtime is good enough to let me think about an *interesting* problem instead. That makes it a legitimate complaint in my eyes. You're welcome to disagree, of course, but do you have an argument more compelling than the one you give here? It seems to me one could use it to argue a lot of Python out of existence. Chief. (Seriously, chief? What are you going for? It sounds like condescension to me; what's the point of that? I hope I'm just misreading you.) Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
exar...@twistedmatrix.com wrote: There's a lot of things in Python that I don't strictly *need*. That doesn't mean that they wouldn't be welcome if I could have them. Getting rid of the range/xrange dichotomy would improve things. The developers agreed a couple of years ago. Starting using 3.1 if you want this. Since 'range' could refer to a user-defined object, rather than the builtin function, there is no way the interpreter should substitute 'xrange'. tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Dr. Phillip M. Feldman wrote: [snip] def is_prime(n): for j in range(2,n): if (n % j) == 0: return False return True It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. You already have an answer to the range issue, so I will only add that putting a loop inside another loop is a decent way to expand the time taken. I will also observe that if you were to stop programming whatever language you are more familiar with in Python, and start programming Python in Python, you'll have an easier time of it. I don't see what's particularly un-Pythonic with this code. Not using xrange() is a mistake, certainly, but it remains clear, easily understandable code which correctly demonstrates the naive algorithm for detecting whether n is a prime. It doesn't call for condescension The Dive Into Python is an excellent start for that. I never read it, but I was under the impression that the tutorial on python.org was geared toward programmers moving to Python from other languages. It was also my impression that Dive Into Python is rather outdated and occasionally inaccurate (based on comments on this mailing list). Cheers, Emm -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, 16 Aug 2009 08:30:54 +0200, Emmanuel Surleau wrote: [...] I will also observe that if you were to stop programming whatever language you are more familiar with in Python, and start programming Python in Python, you'll have an easier time of it. I don't see what's particularly un-Pythonic with this code. Not using xrange() is a mistake, certainly, but it remains clear, easily understandable code which correctly demonstrates the naive algorithm for detecting whether n is a prime. It doesn't call for condescension It's a particular unfair criticism because the critic (Ethan Furman) appears to have made a knee-jerk reaction. The some language in Python behaviour he's reacting to is the common idiom: for i in range(len(seq)): do_something_with(seq[i]) instead of the Python in Python idiom: for obj in seq: do_something_with(obj) That's a reasonable criticism, but *not in the case*. Ethan appears to have made the knee-jerk reaction for i in range() is Bad without stopping to think about what this specific piece of code is actually doing. (Replace 'obj' with 'j', 'seq' with 'range(2, n)', and 'do_something_with' with 'if (n % j) == 0: return False', and you have the exact same code pattern.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
It's a particular unfair criticism because the critic (Ethan Furman) appears to have made a knee-jerk reaction. The some language in Python behaviour he's reacting to is the common idiom: for i in range(len(seq)): do_something_with(seq[i]) instead of the Python in Python idiom: for obj in seq: do_something_with(obj) That's a reasonable criticism, but *not in the case*. Ethan appears to have made the knee-jerk reaction for i in range() is Bad without stopping to think about what this specific piece of code is actually doing. (Replace 'obj' with 'j', 'seq' with 'range(2, n)', and 'do_something_with' with 'if (n % j) == 0: return False', and you have the exact same code pattern.) Fair enough. But as far as I know, for i in (x)range(num) is the canonical way to iterate over numbers in Python. Another case of lack of RTFM* before answering, I suppose. Cheers, Emm *Read The Fine Mail -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, Aug 16, 2009 at 2:30 AM, Emmanuel Surleau emmanuel.surl...@gmail.com wrote: I don't see what's particularly un-Pythonic with this code. Not using xrange() is a mistake, certainly, but it remains clear, easily understandable code which correctly demonstrates the naive algorithm for detecting whether n is a prime. It doesn't call for condescension It's not that the code is bad, but too many people coming from Java and C keep thinking of for loops like they're using Java or C and therefore that for i in range(a,b) is identical to for(int i = a; i b; i++). It's not and, for the most part, you shouldn't code like that. Since you're using numbers, range/xrange is the appropriate idiom but you still have to remember that a for loop in python doesn't loop until a condition is met, it loops through an iterator until the interator says it's done. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Steven D'Aprano st...@remove-this-cybersource.com.au wrote in message news:02969972$0$20647$c3e8...@news.astraweb.com... On Fri, 14 Aug 2009 18:25:45 -0700, Dr. Phillip M. Feldman wrote: It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. Others have already pointed out to you that xrange() (for Python 2.x) does what you want. In Python 3.x, the old range() is gone for good, and xrange() renamed to just range(). It does seem rather worrying that whoever originally thought up Python decided it would be a good idea to implement a simple 1 to N (or 0 to N-1) for-loop by first creating an array of N consecutive integers! They must have known Python was going to be slow anyway, but this wouldn't have helped the speed any. Nor the memory consumption. A for-loop, for iterating over a simple sequence, should be one of the fastest things in the language. [Presumably the internal code that created those consecutive integers used a more conventional looping method...] -- Bartc -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
bartc wrote: Steven D'Aprano st...@remove-this-cybersource.com.au wrote in message news:02969972$0$20647$c3e8...@news.astraweb.com... On Fri, 14 Aug 2009 18:25:45 -0700, Dr. Phillip M. Feldman wrote: It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. Others have already pointed out to you that xrange() (for Python 2.x) does what you want. In Python 3.x, the old range() is gone for good, and xrange() renamed to just range(). It does seem rather worrying that whoever originally thought up Python decided it would be a good idea to implement a simple 1 to N (or 0 to N-1) for-loop by first creating an array of N consecutive integers! Whoever? I am shocked! ;-) They must have known Python was going to be slow anyway, but this wouldn't have helped the speed any. Nor the memory consumption. A for-loop, for iterating over a simple sequence, should be one of the fastest things in the language. [Presumably the internal code that created those consecutive integers used a more conventional looping method...] -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 16 Aug, 11:45, bartc ba...@freeuk.com wrote: A for-loop, for iterating over a simple sequence, should be one of the fastest things in the language. Anyone experienced with interpreted high-level languages knows this is not true. Not because iterating a sequence is expensive, but because the interpreter is constantly executing the body of the loop. There is a reason why Matlab and Python/NumPy programmers rely heavily on vectorized array expressions for efficient numerics. The only thing more evil than a for-loop is recursive function calls. This does not mean Python is slow. It just means you have to program differently. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, Aug 16, 2009 at 6:35 PM, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. but special cases aren't special enough to break the rules. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 17, 8:35 am, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. Extremely easy, once users relinquish the right to replace built-in range with their own concoctions ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On 01:23 am, benjamin.kap...@case.edu wrote: On Sun, Aug 16, 2009 at 6:35 PM, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like � for i in range(n): as a simple integer loop. In fact, Cython is able to do this. but special cases aren't special enough to break the rules. Although I think PyPy also recognizes this case and makes it as efficient as using xrange, and does so without breaking any rules. There *are* _some_ legitimate complaints to be made about the CPython runtime. :) Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
exar...@twistedmatrix.com writes: Although I think PyPy also recognizes this case and makes it as efficient as using xrange, and does so without breaking any rules. How can pypy possibly know that the user hasn't assigned some other value to range? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 16, 6:28 pm, exar...@twistedmatrix.com wrote: On 01:23 am, benjamin.kap...@case.edu wrote: On Sun, Aug 16, 2009 at 6:35 PM, sturlamolden sturlamol...@yahoo.no wrote: A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. In fact, Cython is able to do this. but special cases aren't special enough to break the rules. Although I think PyPy also recognizes this case and makes it as efficient as using xrange, and does so without breaking any rules. PyPy uses a JIT compiler (which is still slower than CPython, suggesting that perhaps they should have spent more time optimizing the general case than optimizing for an easily avoided special case). There *are* _some_ legitimate complaints to be made about the CPython runtime. :) This isn't one of them. xrange has been part of Python for 10 years. If there are any complaints to be made about this situation it's that there are any 2.x learning materials anythere that continue to use range() and not xrange() in this context. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 16, 3:35 pm, sturlamolden sturlamol...@yahoo.no wrote: On 16 Aug, 14:57, Dennis Lee Bieber wlfr...@ix.netcom.com wrote: Well, the alternative would be to have two keywords for looping: one for your simple incrementing integer loop, and another for a loop that operates over the elements of some collection type. A compiler could easily recognise a statement like for i in range(n): as a simple integer loop. It would be a simple to do if you were writing it for a different langauge was a lot less dynamic than Python is. It'd be quite a complex hack to add it to CPython's compiler while maintaing the current highly dynamic runtime semantics and backwards compatibility, which is a design constraint of Python whether you like it or not. And all this complaining about an issue that can be worked around by xrange instead of range. Sheesh. In fact, Cython is able to do this. Cython can do this easily because it is a different language that is a lot less dynamic than Python. If you don't care about the dynamic stuff why don't you just use Cython? Or quit complaining and just use xrange. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Sun, 16 Aug 2009 11:41:21 -0400, Benjamin Kaplan wrote: It's not that the code is bad, but too many people coming from Java and C keep thinking of for loops like they're using Java or C and therefore that for i in range(a,b) is identical to for(int i = a; i b; i++). It's not and, for the most part, you shouldn't code like that. Since you're using numbers, range/xrange is the appropriate idiom but you still have to remember that a for loop in python doesn't loop until a condition is met, it loops through an iterator until the interator says it's done. Java also has iterators; it's more a case of people coming from C and BASIC. Although, some of those may have come *through* Java without abandoning old habits. You see the same thing with people coming from BASIC to C and writing: #define NUM_DATES 50 int day[NUM_DATES], month[NUM_DATES], year[NUM_DATES]; rather than defining a struct. Sometimes referred to as I know ten languages and can write in BASIC in all of them. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
look at xrange -- http://docs.python.org/library/functions.html#xrange -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Saturday 15 August 2009 03:25:45 Dr. Phillip M. Feldman wrote: It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. There is. Use xrange instead of range, and try again. And while you are about it, you may as well teach them that it is much better to do a multiplication than a division. -Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 15, 11:38 am, Mark Lawrence breamore...@yahoo.co.uk wrote: Dr. Phillip M. Feldman wrote: I wrote the following correct but inefficient test of primality for purposes of demonstrating that the simplest algorithm is often not the most efficient. But, when I try to run the following code with a value of n that is large enough to produce a significant amount of running time, I get an out-of-memory error! def is_prime(n): for j in range(2,n): if (n % j) == 0: return False return True It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. I have a strong suspicion that you will find hints in the Python documentation that this has already been addressed. Perhaps you could try reading before posting? Alternative hypothesis: the good doctor read the Python 3.x documentation but absent-mindedly ran Python 2.x -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Fri, 14 Aug 2009 18:25:45 -0700, Dr. Phillip M. Feldman wrote: It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. Others have already pointed out to you that xrange() (for Python 2.x) does what you want. In Python 3.x, the old range() is gone for good, and xrange() renamed to just range(). However, I'd like to point out that your subject line is fundamentally incorrect. What you've discovered has *nothing* to do with for-loops: the for-loop will happily iterate over whatever object you pass to it (or at least try to, because not all objects are iterable). You might be thinking that Python requires you to write for-loops as for i in range(...): but that's not correct. The for-loop doesn't care what object comes after the in and before the :, so long as it can iterate over it one item at a time: for i in [1, 3, 4, 5, 2, 0, -1, 8, 7, 6]: print i for c in abcd: print c and many other variations will work perfectly fine. The memory-consumption you have discovered is a property of the range() function, not the for-loop: L = range(2, 20) L [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] Compare to xrange: L = xrange(2, 20) L xrange(2, 20) list(L) # expand out into a list [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] By the way, both range() and xrange() take an optional 'stepsize' argument, defaulting to 1: range(3, 20, 2) [3, 5, 7, 9, 11, 13, 15, 17, 19] help(range) and help(xrange) in the interactive interpreter are your friends. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Hendrik van Rooyen wrote: On Saturday 15 August 2009 03:25:45 Dr. Phillip M. Feldman wrote: And while you are about it, you may as well teach them that it is much better to do a multiplication than a division. Actually, division speed hasn't been much of an issue in years. Arithmetic has been faster than memory access since CPUs went superscalar. In CPython, with all the interpreter overhead, you'll probably never notice the difference. I'd thought xrange had already been obsoleted, and hadn't realized the 2.3 - 2.6 versions of CPython were still doing range the hard way, not as a generator. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
John Nagle wrote: Hendrik van Rooyen wrote: On Saturday 15 August 2009 03:25:45 Dr. Phillip M. Feldman wrote: And while you are about it, you may as well teach them that it is much better to do a multiplication than a division. Actually, division speed hasn't been much of an issue in years. Arithmetic has been faster than memory access since CPUs went superscalar. In CPython, with all the interpreter overhead, you'll probably never notice the difference. I'd thought xrange had already been obsoleted, and hadn't realized the 2.3 - 2.6 versions of CPython were still doing range the hard way, not as a generator. It would've broken some existing code, hence it was left until Python 3. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. This has nothing to do with Python's for loop (and saying the loop construct itself is memory inefficient makes me blink in confusion): but you've isolated the problem precisely all on your own. range() is defined as returning a list. Thus, it constructs the list all at once. xrange() returns an iterator, and generates the values on the fly. In Python 3, this was changed so range returns an iterator, and to get a list you do list(range(2,n)). --S -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Dr. Phillip M. Feldman wrote: I wrote the following correct but inefficient test of primality for purposes of demonstrating that the simplest algorithm is often not the most efficient. But, when I try to run the following code with a value of n that is large enough to produce a significant amount of running time, I get an out-of-memory error! def is_prime(n): for j in range(2,n): if (n % j) == 0: return False return True It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. I have a strong suspicion that you will find hints in the Python documentation that this has already been addressed. Perhaps you could try reading before posting? -- Kindest regards. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
Dr. Phillip M. Feldman wrote: I wrote the following correct but inefficient test of primality for purposes of demonstrating that the simplest algorithm is often not the most efficient. But, when I try to run the following code with a value of n that is large enough to produce a significant amount of running time, I get an out-of-memory error! def is_prime(n): for j in range(2,n): if (n % j) == 0: return False return True It seems as though Python is actually expanding range(2,n) into a list of numbers, even though this is incredibly wasteful of memory. There should be a looping mechanism that generates the index variable values incrementally as they are needed. You already have an answer to the range issue, so I will only add that putting a loop inside another loop is a decent way to expand the time taken. I will also observe that if you were to stop programming whatever language you are more familiar with in Python, and start programming Python in Python, you'll have an easier time of it. The Dive Into Python is an excellent start for that. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 'for' loop is memory inefficient
On Aug 14, 8:25 pm, Dr. Phillip M. Feldman pfeld...@verizon.net wrote: I wrote the following correct but inefficient test of primality for purposes of demonstrating that the simplest algorithm is often not the most efficient. But, when I try to run the following code with a value of n that is large enough to produce a significant amount of running time, I get an out-of-memory error! I don't think Python was created to keep mathematician's happy. Although strangely enough, the creator holds a masters degree in mathematics *and* computer science. Just another one of life's little ironies ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article pecora-d381e9.08361703042...@ra.nrl.navy.mil, Lou Pecora pec...@anvil.nrl.navy.mil wrote: In article 5c92e9bd-1fb4-4c01-a928-04d7f6733...@e21g2000yqb.googlegroups.com, Aaron Brady castiro...@gmail.com wrote: Did I tell you guys that 'natural' has 38 definitions at dictionary.com? Amazing. I suggest you pick the one that fits best. The wonderful thing about standards is that there are so many to choose from. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ ...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons. --Aahz -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 2, 5:29 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: On Wed, 01 Apr 2009 21:58:47 -0700, Lie wrote: On Apr 1, 7:06 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. No, there won't be any bizarro-talk. There is no argument: the zeroeth item comes zeroeth, the first item comes first, and so on. The index for the very zeroeth thing in a list is 0, so to get the zeroeth item you use s[0]. While to get the first item you use s[1]. It's very intuitive, isn't it? No, because first, second, third etc. have existed in the English language for hundreds of years and everybody knows them. Zeroeth was probably invented a few decades ago, and is known by maybe 1% of the English-speaking population. Given the list [a, b, c], if you ask even a C programmer *in English* what's the first item?, they will almost invariably answer a rather than b. -- Steven Alternatively: One friend of mine half-seriously advanced the following thesis: We should count from zero. But first is, etymologically, a diminution of foremost, and (as TomStambaugh says) should mean 0th when we count from 0. And second is from the Latin secundus, meaning following, so it should mean 1th when we count from 0. Obviously the ordinals from third onwards get their names from the numbers. So... we need a new word for 2th. He proposed twifth. Thus: first, second, twifth, third, fourth, ... -- GarethMcCaughan (from http://c2.com/cgi/wiki?ZeroAndOneBasedIndexes ) No zeroeth, twifth -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article 5c92e9bd-1fb4-4c01-a928-04d7f6733...@e21g2000yqb.googlegroups.com, Aaron Brady castiro...@gmail.com wrote: On Apr 2, 6:34 pm, Tim Wintle tim.win...@teamrubber.com wrote: On Thu, 2009-04-02 at 15:16 -0700, Emile van Sebille wrote: Lou Pecora wrote: Confusion only comes when you try to force the defintion of one of them on the other and then say it's illogical or not natural. Both are natural. Consider the French 'Premiere etage' vs the American 'First Floor' or even in the same language - first floor in English (UK) is very different from first floor in English (US). Did I tell you guys that 'natural' has 38 definitions at dictionary.com? Amazing. I suggest you pick the one that fits best. -- -- Lou Pecora -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Lie wrote: [snip] Alternatively: One friend of mine half-seriously advanced the following thesis: We should count from zero. But first is, etymologically, a diminution of foremost, and (as TomStambaugh says) should mean 0th when we count from 0. And second is from the Latin secundus, meaning following, so it should mean 1th when we count from 0. Obviously the ordinals from third onwards get their names from the numbers. So... we need a new word for 2th. He proposed twifth. Thus: first, second, twifth, third, fourth, ... I propose twoth (sounds like tooth). :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 3, 10:36 pm, Lou Pecora pec...@anvil.nrl.navy.mil wrote: Aaron Brady castiro...@gmail.com wrote: Did I tell you guys that 'natural' has 38 definitions at dictionary.com? Amazing. I suggest you pick the one that fits best. You mean the one that feels most natural? -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 3, 10:43 am, alex23 wuwe...@gmail.com wrote: On Apr 3, 10:36 pm, Lou Pecora pec...@anvil.nrl.navy.mil wrote: Aaron Brady castiro...@gmail.com wrote: Did I tell you guys that 'natural' has 38 definitions at dictionary.com? Amazing. I suggest you pick the one that fits best. You mean the one that feels most natural? No, not feels. *Is*. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 2, 4:05 pm, Aaron Brady castiro...@gmail.com wrote: On Apr 1, 11:58 pm, Lie lie.1...@gmail.com wrote: On Apr 1, 7:06 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. No, there won't be any bizarro-talk. There is no argument: the zeroeth item comes zeroeth, the first item comes first, and so on. The index for the very zeroeth thing in a list is 0, so to get the zeroeth item you use s[0]. While to get the first item you use s[1]. It's very intuitive, isn't it? Robot 1: I won zeroeth place at the contest, honey! Robot 2: Congratulations! I knew you could do it. That should be Robot 0 and Robot 1. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Thu, 02 Apr 2009 04:23:32 +, John O'Hagan wrote:
Beyond being part of a conventionally-ordered set of keys, what can an
ordinality of zero actually mean? (That's a sincere question.)
In set theory, you start by defining the integers like this:
0 is the cardinality (size) of the empty set, the set with nothing in it.
1 is the cardinality of the set of empty sets, that is, the set
containing nothing but the empty set.
2 is the cardinality of the set of the empty set plus the set of empty
sets.
3 is the cardinality of the set containing the empty set, plus the set of
empty sets, plus the set of (the empty set plus the set of empty sets).
And so forth, to infinity and beyond.
Or to put it another way:
0 = len( {} )
1 = len( {{}} )
2 = len( {{}, {{}}} )
3 = len( {{}, {{}}, {{}, {{}}} )
etc.
For non-infinite sets, you can treat ordinal numbers and cardinal numbers
as more or less identical. So an ordinality of zero just means the number
of elements of something that doesn't exist.
How that relates to whether indexing should start at one or zero, I have
no idea.
Oh, and speaking of... I'm shocked, SHOCKED I say, that nobody has given
that quote about the compromise of 0.5.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Wed, 01 Apr 2009 21:58:47 -0700, Lie wrote: On Apr 1, 7:06 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. No, there won't be any bizarro-talk. There is no argument: the zeroeth item comes zeroeth, the first item comes first, and so on. The index for the very zeroeth thing in a list is 0, so to get the zeroeth item you use s[0]. While to get the first item you use s[1]. It's very intuitive, isn't it? No, because first, second, third etc. have existed in the English language for hundreds of years and everybody knows them. Zeroeth was probably invented a few decades ago, and is known by maybe 1% of the English-speaking population. Given the list [a, b, c], if you ask even a C programmer *in English* what's the first item?, they will almost invariably answer a rather than b. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 1, 9:23 pm, John O'Hagan resea...@johnohagan.com wrote: Despite being thoroughly acclimatised to zero-based indexing and having no wish to change it, I'm starting to see the OP's point. Many of the arguments presented in this thread in favour of zero-based indexing have rather been arguments for half-open intervals, which I don't think are in dispute. We all want these to be true: foo[:n] is the first n items of the sequence foo foo[:n] + foo[n:] == foo len(foo[n:m]) == m-n (foo[n:n]) is an empty sequence etc. and they are true with 0-based indexing if we exclude the last number, or equally with 1-based indexing if we exclude the first. Unless I'm missing something, wouldn't that mean: range(0,10) - [1,2,3,4,5,6,7,8,9,10] Even though it's theoretically just another way to line up the open interval, as practical matter it's going to be a lot more confusing. Of course you could exclude the last number with one-based indexing also, but that would be confusing too, since you would have to use something like range(1,len(x)+1) to iterate over the items of x. Given that, I'm going to disagree that a half-open interval is desirable in the case of one-based indexing. Furthermore, I know of no languages that use both one-based indexing and half-open intervals. Do you know of any? Beyond being part of a conventionally-ordered set of keys, what can an ordinality of zero actually mean? (That's a sincere question.) I think people were being facetious. To me the first item in the list is x[0]--ordinal does not match cardinal. However, I don't use ordinals much when talking about list items; I'll say item 2, not third item. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 2, 1:29 am, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: On Wed, 01 Apr 2009 21:58:47 -0700, Lie wrote: On Apr 1, 7:06 pm, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. No, there won't be any bizarro-talk. There is no argument: the zeroeth item comes zeroeth, the first item comes first, and so on. The index for the very zeroeth thing in a list is 0, so to get the zeroeth item you use s[0]. While to get the first item you use s[1]. It's very intuitive, isn't it? No, because first, second, third etc. have existed in the English language for hundreds of years and everybody knows them. Zeroeth was probably invented a few decades ago, and is known by maybe 1% of the English-speaking population. Given the list [a, b, c], if you ask even a C programmer *in English* what's the first item?, they will almost invariably answer a rather than b. -- Steven However, if you ask him/er, What is the item that is 0 items from the start of the list?, what will s/he say? -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Carl Banks pavlovevide...@gmail.com writes: This is unforgiveable, not only changing the indexing semantics of Python (because a user would have NO CLUE that something underlying has been changed, and thus it should never be done), but also for the needless abuse of exec. Then I guess you'd fire Guido, too -- from socket.py: class _socketobject(object): [...] _s = (def %s(self, *args): return self._sock.%s(*args)\n\n %s.__doc__ = _realsocket.%s.__doc__\n) for _m in _socketmethods: exec _s % (_m, _m, _m, _m) del _m, _s -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Carl Banks wrote: On Apr 1, 2:32 pm, Arnaud Delobelle arno...@googlemail.com wrote: Check the date on the line above (and the PS in that post). If I were your boss and you ever pulled something like this, your ass would be so fired. This is unforgiveable, not only changing the indexing semantics of Python (because a user would have NO CLUE that something underlying has been changed, and thus it should never be done), but also for the needless abuse of exec. Gotcha ;) -- Arnaud PS. I disagree about the 'needless abuse of exec'. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 1, 11:28 pm, Hrvoje Niksic hnik...@xemacs.org wrote: Carl Banks pavlovevide...@gmail.com writes: This is unforgiveable, not only changing the indexing semantics of Python (because a user would have NO CLUE that something underlying has been changed, and thus it should never be done), but also for the needless abuse of exec. Then I guess you'd fire Guido, too -- from socket.py: class _socketobject(object): [...] _s = (def %s(self, *args): return self._sock.%s(*args)\n\n %s.__doc__ = _realsocket.%s.__doc__\n) for _m in _socketmethods: exec _s % (_m, _m, _m, _m) del _m, _s Damn straight I would, recklessly using exec like he owns the language or something. Carl Bansk -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Thu, 2 Apr 2009, Steven D'Aprano wrote: On Thu, 02 Apr 2009 04:23:32 +, John O'Hagan wrote: Beyond being part of a conventionally-ordered set of keys, what can an ordinality of zero actually mean? (That's a sincere question.) [snip erudite definition of cardinality] For non-infinite sets, you can treat ordinal numbers and cardinal numbers as more or less identical. So an ordinality of zero just means the number of elements of something that doesn't exist. This is the bit I don't get - I had thought of ordinality as something attached to each item - ['a','b','c'] has a cardinality of 3, and elements of ordinality 1, 2 and 3 (first,second, third) respectively. So it's possible to have a cardinality of zero (an empty sequence does) but only something that doesn't exist can have an ordinality of zero; as soon as there is an item, its ordinality is 1. Shoot me down, please! How that relates to whether indexing should start at one or zero, I have no idea. Only insofar as the weirdness of indexing being out of step with ordinality/cardinality matters, i.e. not that much. :) John -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Steven D'Aprano wrote:
In set theory, you start by defining the integers like this:
0 is the cardinality (size) of the empty set, the set with nothing in it.
1 is the cardinality of the set of empty sets, that is, the set
containing nothing but the empty set.
2 is the cardinality of the set of the empty set plus the set of empty
sets.
3 is the cardinality of the set containing the empty set, plus the set of
empty sets, plus the set of (the empty set plus the set of empty sets).
And so forth, to infinity and beyond.
Or to put it another way:
0 = len( {} )
1 = len( {{}} )
2 = len( {{}, {{}}} )
3 = len( {{}, {{}}, {{}, {{}}} )
FWIW this is the way I learnt it AFAIK:
Ordinals
===
0 *is* the empty set
1 *is* the the the singleton composed of the empty set, i.e. {0}
2 *is* the set {0, 1}
3 *is* the set {0, 1, 2}
...
n + 1 := n U {n}
It's nice because:
* the interval [0, n) is just the number n
* n m iff n is a subset of m iff n is a member of m
Cardinals
=
A cardinal is an equivalence class under the equivalence relation S ~
S' iff there is a bijection between S and S'. Obviously, finite
cardinals contain only one ordinal so finite cardinals can be
identified with their ordinal representative.
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
En Wed, 01 Apr 2009 08:04:12 -0300, andrew cooke and...@acooke.org escribió: something i don't think has been mentioned much - if you're using range() in your python code then you're almost always doing it wrong. i just grepped lepl and i use range 20 times in 9600 lines of code. out of those, all but 3 are in quick and dirty tests or experimental code, not in the main library itself (which is 6300 lines). (1) where i need to access two adjacent members of a list, and which has a comment in the code explaining why it is not an error (in other words, i was so unhappy with my code i needed to leave a note explaining why it was like that) From your description I guess this range usage could have been avoided, using enumerate instead. A silly example: for i,elem in enumerate(some_list): if i0: prev = some_list[i-1] print (elem+prev)/2 instead of: for i in range(1, len(some_list)): elem = some_list[i] prev = some_list[i-1] print ... (2) a use irrelevant to this discussion because i do not use the value to an index an array. (3) in the rather complex implementation of a circular buffer. I can't tell, but perhaps enumerate() could have been used here too? so in a small/moderate size library of 6000 lines (including blanks and comments, but excluding tests and exploratory code) the only time i have used range with array indices i was either unhappy with the code, or implementing a complex data structure. Maybe the ratio is even less than that. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article pan.2009.04.02.06.28...@remove.this.cybersource.com.au, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: So an ordinality of zero just means the number of elements of something that doesn't exist. You do realize that will give most people headaches. :-) -- -- Lou Pecora -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Thu, 2009-04-02 at 06:28 +, Steven D'Aprano wrote:
In set theory, you start by defining the integers like this:
snip
0 = len( {} )
1 = len( {{}} )
2 = len( {{}, {{}}} )
3 = len( {{}, {{}}, {{}, {{}}} )
etc.
not quite len() - surely you mean something like any object along with
an algebra in which the left hand side is equivalent to the right in the
algebra of set theory? - at least for ordinals.
The cardinal is then (for finite numbers) the length of that.
Or, in a pythonic sense (taking 0,1,2,... to be variable names):
0 = set()
1 = set(0)
2 = set(1,0)
3 = set(2,1,0)
3 = set(3,2,1,0)
etc.
How that relates to whether indexing should start at one or zero, I
have
no idea.
so in this sense, range(n) is actually very close to the ordinal value
of 0 (except for being a list and not a set - but it's not in 3.0)
i.e. range(n) returns something very similar to the ordinal n, and
with cardinality n. That seems very sensible to me.
Oh, and speaking of... I'm shocked, SHOCKED I say, that nobody has
given that quote about the compromise of 0.5.
God made the integers, all else is the work of man - Leopold Kronecker
...holding myself back from complaining about integer division in Py3K
when the philosophical question of whether irrational numbers even exist
(in a physics sense) is fairly open.
Tim W
--
http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article bd70785c-1e86-4437-a14e-d84028f57...@k19g2000prh.googlegroups.com, Carl Banks pavlovevide...@gmail.com wrote: I think people were being facetious. To me the first item in the list is x[0]--ordinal does not match cardinal. However, I don't use ordinals much when talking about list items; I'll say item 2, not third item. Well, it's been said in many forms in this thread, but maybe this version will click with some. Thinking of ordinals as specifying position by order and cardinals as counting the number of steps from a specified place to an item you just have to realize that 0 and 1 based indexing have decided to use *different* definitions for the conglomerate symbols A[i] or A(i): 1 based indexing (ordinality): A[i] is the ith item in the ordered list (1st, 2nd, etc. - no need for 0th as an ordinal) 0 based indexing (cardinality): A[i] is the item i steps from the beginning of the list. I know most of what's been said is all around the above, I just thought explict statements might help. Both are prefectly reasonable and consistent definitions. Confusion only comes when you try to force the defintion of one of them on the other and then say it's illogical or not natural. Both are natural. Of course, in some languages like C an array name, say A, points to the first item in memory of the array which is assumed to be structured sequentially. Then A[0] is the first item at the address A or since we are using cardinality (0 based indexing) it's the item 0 steps from the beginning. A[1] is the item at address A+1, i.e. 1 step from the address A, the array beginning. I suspect that was a very practical choice for C since it's a systems language and down at that level you're flipping bit, bytes, addresses, etc. But a practical consequence was that there was very little +1 or -1 arithmetic necessary to manipulate the array elements. That carried over into other languages and their design. People have already pointed out the nice features of such in Python where A[n:n] is an empty list, A[:n]+A[n:]=A, etc. Now, I'm not a systems/computer guy. Just a scientist who does a lot of number crunching. But I have found out that even in that case you end of traversing lists, tuples, etc. a lot. Slicing and dicing them. And you quickly come to appreciate the 0 based approach to indexing and soon don't miss the Fortran 1-based indexing. -- -- Lou Pecora -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Lou Pecora wrote: Confusion only comes when you try to force the defintion of one of them on the other and then say it's illogical or not natural. Both are natural. Consider the French 'Premiere etage' vs the American 'First Floor' Emile -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Thu, 2009-04-02 at 15:16 -0700, Emile van Sebille wrote: Lou Pecora wrote: Confusion only comes when you try to force the defintion of one of them on the other and then say it's illogical or not natural. Both are natural. Consider the French 'Premiere etage' vs the American 'First Floor' or even in the same language - first floor in English (UK) is very different from first floor in English (US). Emile -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Apr 2, 6:34 pm, Tim Wintle tim.win...@teamrubber.com wrote: On Thu, 2009-04-02 at 15:16 -0700, Emile van Sebille wrote: Lou Pecora wrote: Confusion only comes when you try to force the defintion of one of them on the other and then say it's illogical or not natural. Both are natural. Consider the French 'Premiere etage' vs the American 'First Floor' or even in the same language - first floor in English (UK) is very different from first floor in English (US). Did I tell you guys that 'natural' has 38 definitions at dictionary.com? -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Chris Rebert c...@rebertia.com writes: Among other things, it has the nice property that: len(some_list[n:m]) == m-n And also that it is intuitive how to represent an empty slice (foo[n:n]). When traversing over sublists, it's also a useful property that foo[a:b] + foo[b:c] == foo. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Mar 31, 7:23 pm, Lada Kugis lada.ku...@gmail.com wrote: On 01 Apr 2009 01:26:41 GMT, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: Why Python (and other languages) count from zero instead of one, and why half-open intervals are better than closed intervals: http://www.johndcook.com/blog/2008/06/26/why-computer-scientists-coun... http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html steven, thanks for answering, yes, i saw the second one a little time ago (someone else posted it as well in really cute handwriting version :) and the first time just now, but the examples which both of them give don't seem to me to be that relevant, e.g. the pros don't overcome the cons. imho, although both sides (mathematical vs engineer) adress some points, none of them give the final decisive argument. i understand the math. point of view, but from the practical side it is not good. it goes nicely into his tidy theory of everything, but practical and intuitive it is not. as i said, being an engineer, i tend towards the other side, Lada, I am also an engineer, and I can tell your idea of intuitive is not universal, even among engineers. I certainly do not lean toward one- based indexing. From a programming standpoint--and remember Python is a programming language--zero-based indexing eliminates the need for a whole lot of extra +1s and -1s when indexing, slicing, and iterating, a lot more than it causes, and that is worth the cost. This might not be apparent to you if you never tried seriously taking advantage of indexing from zero, or if your programming experience is very narrow. These both seem to be true for you, so you'll just have to take my word for it. I'm sorry that the zero-based indexing isn't the most natural for your field. There are some things about Python that aren't the natural choice for me, too. I would, however, highly recommend that you avoid the ridiculous pseudo-one-based indexing trick you pulled when interfacing C to Fortran. Python is zero-based, and you are going to have a much better experience if you don't fight it. I assure you that it is really not that hard to cope with indices being off by one from what you have written down. Really. I have to interface two languages, one of which indexes from one, the other from zero, and it really is no problem. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Lada Kugis schrieb: On 01 Apr 2009 01:26:41 GMT, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: Why Python (and other languages) count from zero instead of one, and why half-open intervals are better than closed intervals: http://www.johndcook.com/blog/2008/06/26/why-computer-scientists-count-from-zero/ http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html steven, thanks for answering, yes, i saw the second one a little time ago (someone else posted it as well in really cute handwriting version :) and the first time just now, but the examples which both of them give don't seem to me to be that relevant, e.g. the pros don't overcome the cons. imho, although both sides (mathematical vs engineer) adress some points, none of them give the final decisive argument. i understand the math. point of view, but from the practical side it is not good. it goes nicely into his tidy theory of everything, but practical and intuitive it is not. You keep throwing around the concept of intuition as if it were something that existis in a globally fixed frame of reference. It is not. From [1]: Klein found that under time pressure, high stakes, and changing parameters, experts used their base of experience to identify similar situations and intuitively choose feasible solutions. In other words: your gained *knowledge* (including mathematical know-how) determines what's intuitive - to *you*. So all arguing about intuition is rather irrelevant - to me, zero-based counting is intuitive, and it is so with the same right as you prefer one-based. OTOH zero-based counting has technical reasons that also reduces the amount of leaky abstraction problems [2], as it models the underlying reality of memory locations + indexing. Diez [1] http://en.wikipedia.org/wiki/Intuition_(knowledge) [2] http://www.joelonsoftware.com/articles/LeakyAbstractions.html -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Wed, 01 Apr 2009 04:39:26 +0100, Rhodri James wrote: Dragging this back to the original topic, you clearly find starting list indices from zero unintuitive. To me, with a mathematical background, it's not just intuitive, it's correct. All sorts of useful properties fall out from that, not the least of which is the fact that a[0:len(a)] slices the whole of a list. But some non-useful properties fall out of that too. Want the fifth item? a[5] gives you the *sixth* item, which is weird, so you have to use a[5-1] to get the fifth item. There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. a[0:len(a)] is legal, a[0] is legal, but surprisingly a[len(a)] is an error. Despite coming from a Pascal background, I've come to appreciate and prefer zero-based indexing for programming. But I'm not blind to the disadvantages. I'll often work out an algorithm using pencil and paper and counting from one, and then subtract one to get zero-based indexes. There are advantages and disadvantages to both systems, but on balance, I think that zero-based is a better system for programming, and one-based for natural language. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
something i don't think has been mentioned much - if you're using range() in your python code then you're almost always doing it wrong. i just grepped lepl and i use range 20 times in 9600 lines of code. out of those, all but 3 are in quick and dirty tests or experimental code, not in the main library itself (which is 6300 lines). so in my main code, i use range() once in every 2000 lines of code, approximately. the three examples are: (1) where i need to access two adjacent members of a list, and which has a comment in the code explaining why it is not an error (in other words, i was so unhappy with my code i needed to leave a note explaining why it was like that) (2) a use irrelevant to this discussion because i do not use the value to an index an array. (3) in the rather complex implementation of a circular buffer. so in a small/moderate size library of 600 lines (including blanks and comments, but excluding tests and exploratory code) the only time i have used range with array indices i was either unhappy with the code, or implementing a complex data structure. andrew -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
andrew cooke wrote: [...] so in a small/moderate size library of 600 lines (including blanks and 6000 comments, but excluding tests and exploratory code) the only time i have used range with array indices i was either unhappy with the code, or implementing a complex data structure. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Natural language is full of ambiguity, which is why my parents used to argue about the meaning of next Wednesday, or of the next exit. Until you have a starting reference, and until you decide whether it's a closed or open interval, you can't be sure everyone will get the same semantics. Because of their bickering 50 years ago, I try to use a longer phrase, such as the Wednesday a week from today or exit 52. In most? of Europe, the first floor is the one you get to after you go up one flight of stairs. In the US, first floor is usually the ground floor. My ruler starts at zero, and has an explicit zero on it. Most rulers start the same way, but don't actually show the zero. But if you had a ruler that started with one, you'd end up with some off-by-one errors. I repeatedly see people think that if Joe has $100, and Mary has three times more, that she has $300. Yet these same people would balk at the notion that 10% more would be $10. English is a funny language (as I suspect most languages are), and full of ambiguities and foolish inconsistencies. Having worked with some 35 languages over 35 years in the industry, I was glad that most of them were zero based. Steven D'Aprano wrote: On Wed, 01 Apr 2009 04:39:26 +0100, Rhodri James wrote: Dragging this back to the original topic, you clearly find starting list indices from zero unintuitive. To me, with a mathematical background, it's not just intuitive, it's correct. All sorts of useful properties fall out from that, not the least of which is the fact that a[0:len(a)] slices the whole of a list. But some non-useful properties fall out of that too. Want the fifth item? a[5] gives you the *sixth* item, which is weird, so you have to use a[5-1] to get the fifth item. There is a major clash between the names of ordinals in human languages and zero-based counting. In human languages, the Nth-ordinal item comes in position N. You can keep that useful convention with zero-based counting by inventing the ugly word zeroth, but that just leads to bizarro-talk like the zeroeth item comes first, the first item comes second, and so on. a[0:len(a)] is legal, a[0] is legal, but surprisingly a[len(a)] is an error. Despite coming from a Pascal background, I've come to appreciate and prefer zero-based indexing for programming. But I'm not blind to the disadvantages. I'll often work out an algorithm using pencil and paper and counting from one, and then subtract one to get zero-based indexes. There are advantages and disadvantages to both systems, but on balance, I think that zero-based is a better system for programming, and one-based for natural language. -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Lada Kugis: (you have 1 apple, you start counting from 1 ... To little children I now show how to count starting from zero: apple number zero, apple number one, etc, and they find it natural enough :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Wed, 01 Apr 2009 04:39:39 -0700, bearophileHUGS wrote: Lada Kugis: (you have 1 apple, you start counting from 1 ... To little children I now show how to count starting from zero: apple number zero, apple number one, etc, and they find it natural enough :-) Ah, but that's not the same as counting ordinals from zero. Consider the difference between an empty list, which has zero items, and a list of length one, which as that item in position zero. The empty list [] clearly has zero items, in zero bins, so the bins aren't numbered. A list with one item [x] has one bin, labelled Index 0. A list with two items [x, y] has two bins, labelled Index 0 and Index 1. A list with three items [x, y, z] has three bins, labelled Index 0, Index 1 and Index 2. etc. Counting No apples plus one apple makes one apple is easy enough. But pointing to an apple clearly there and saying Let's call that apple number zero, even though there is one apple, that requires a very special child! -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
2009/4/1 Carl Banks pavlovevide...@gmail.com: I am also an engineer, and I can tell your idea of intuitive is not universal, even among engineers. I certainly do not lean toward one- based indexing. Another engineer here who finds 0-based indexing more intuitive than 1-based indexing. -- Tim Rowe -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article 1ej5t4930m29h0f6ttpdcd83t08q2q3...@4ax.com, Lada Kugis lada.ku...@gmail.com wrote: On 01 Apr 2009 01:26:41 GMT, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: Why Python (and other languages) count from zero instead of one, and why half-open intervals are better than closed intervals: http://www.johndcook.com/blog/2008/06/26/why-computer-scientists-count-from-z ero/ http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html steven, thanks for answering, yes, i saw the second one a little time ago (someone else posted it as well in really cute handwriting version :) and the first time just now, but the examples which both of them give don't seem to me to be that relevant, e.g. the pros don't overcome the cons. imho, although both sides (mathematical vs engineer) adress some points, none of them give the final decisive argument. i understand the math. point of view, but from the practical side it is not good. it goes nicely into his tidy theory of everything, but practical and intuitive it is not. as i said, being an engineer, i tend towards the other side, so this is biased opinion (nobody can be unbiased) but from a practical side it seems unpractical for engineering problems (and to me, the purpose of computers is to help humans to build a better world, not to prove theories - theories are useless if they don't help us in reality. so we should try to adapt computing to real world, not our world to computers). There is nothing more practical than a good theory. --- James Clerk Maxwell You said you came from the C world (besides Fortran). If so, you have already seen array indexing starting with 0 and going to n-1. Why, then, should Python be so foreign to you? -- -- Lou Pecora -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
In article 72i5t4tgfo2h4gd6ggcs02flkca85kg...@4ax.com, Lada Kugis lada.ku...@gmail.com wrote: and the (m-n) point Chris was trying to explain doesn't seem that relevant to me. That's because you haven't done enough programming really using the Python structures and objects. You can really do a lot with lists and tuples. When you do you will see Chris' point emphatically. -- -- Lou Pecora -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On Wed, 1 Apr 2009 00:40:17 -0700 (PDT), Carl Banks pavlovevide...@gmail.com wrote: Lada, I am also an engineer, and I can tell your idea of intuitive is not universal, even among engineers. I certainly do not lean toward one- based indexing. From a programming standpoint--and remember Python is a programming language--zero-based indexing eliminates the need for a whole lot of extra +1s and -1s when indexing, slicing, and iterating, a lot more than it causes, and that is worth the cost. This might not be apparent to you if you never tried seriously taking advantage of indexing from zero, or if your programming experience is very narrow. These both seem to be true for you, so you'll just have to take my word for it. You have repeated several cs based points already stated. But apart from a programming standpoint - could you give a few examples, where on paper (as to avoid stepping into programmer's territory) zero indexing could be more intuitive ? (of course, taking into account your previous based calculations, which are based on 1 indexing - I imagine you still use matrices with a11 as a first element) Lada -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
On 01 Apr 2009 08:06:28 GMT, Steven D'Aprano ste...@remove.this.cybersource.com.au wrote: There are advantages and disadvantages to both systems, but on balance, I think that zero-based is a better system for programming, and one-based for natural language. Nicely put. Yes, along with some of your other arguments, I think I can agree that this sums it up best. I'll just have to adapt myself to natural language thinking at one side, and programming thinking at the other. Lada -- http://mail.python.org/mailman/listinfo/python-list
Re: python for loop
Lada Kugis wrote: [snip] Yes, that's it. I won't argue over it, since I can't change it, but 1 is still more natural to me (it is the real world way). The above pros seem to me to be very little compared to the cons of the ... and the (m-n) point Chris was trying to explain doesn't seem that relevant to me. I always thought since python is trying so hard to be as intuitive as it can get it would be the other way (you have 1 apple, you start counting from 1 ... [snip] But you might not have any apples, so you should start from 0! :-) -- http://mail.python.org/mailman/listinfo/python-list
