Re: range() is not the best way to check range?
On 2006-07-23, Paul Boddie [EMAIL PROTECTED] wrote: Antoon Pardon wrote: Except that if you write your own class from scratch, you can't use it as a slice. Correct, but we were actually discussing subclassing built-in classes for use as a replacement for range/xrange. :-) I think a slice could be very usefull for that. To me it feel very natural that a slice would be iterable and could be used in a form like: for i in slice(10,20): or if the slice notation would be usable for i in (10:20): And why not make a slice indexable too? Whether slices really are the tool to use here or not I don know for sure. But the problem is, that you can't easily experiment with this idea because slices are not subclassable. It may be hard work writing all those methods in a totally new range/xrange class, but passing objects of that class around should prove satisfactory for the use of most programs. But the parameters you will have to give such a class are the same you have to pass to a slice. IMO that means that you should at least consider giving slices this functionality. At least it should be easy to convert from one to the other and back. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-21, Paul Boddie [EMAIL PROTECTED] wrote: Regardless of whether myslice inherits from object or not, there's no persuading the interpreter that it is a genuine slice, and remember that we can't subclass slice (for some reason unknown). So, it would appear that the interpreter really wants instances from some specific set of types (presumably discoverable by looking at list_subscript in listobject.c) rather than some objects conforming to some interface or protocol, and perhaps it is implemented this way for performance reasons. In any case, in the core of Python some types/classes are more equal than others, and for whatever reason the duck typing breaks down - a case of malbik endar [1] that you just have to be aware of, I suppose. Is there any chance this will iever change? Is there any chance the start:stop:step notation will ever be considered an atom? -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Antoon Pardon wrote: Except that if you write your own class from scratch, you can't use it as a slice. Correct, but we were actually discussing subclassing built-in classes for use as a replacement for range/xrange. :-) It may be hard work writing all those methods in a totally new range/xrange class, but passing objects of that class around should prove satisfactory for the use of most programs. I personally doubt that it is that much hard work, especially if you stick to a reasonable selection of list capabilities, for example, rather than attempting to emulate support for every dodgy trick available to the programmer in the modern CPython arsenal. For a language that is supposed to be about duck typing I find it strange that if I make my own class with a start, stop and step attribute, that python barfs on it when I want to use it as a slice. Yes, my post showed this and gave a reference to where in the CPython source code the tests for specific types are performed. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-20, Paul Boddie [EMAIL PROTECTED] wrote: Alex Martelli wrote: Paul Boddie [EMAIL PROTECTED] wrote: Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). You'd also have to override just about every mutating method to switch back to a normal __contains__ (or change self's type on the fly) -- a pretty heavy price to pay. A subclass of list is probably a bad idea in hindsight, due to various probable requirements of it actually needing to be a list with all its contents, whereas we wanted to avoid having anything like a list around until the contents of this lazy list were required by the program. If we really wanted to subclass something, we could consider subclassing the slice class/type, but that isn't subclassable in today's Python for some reason, and it doesn't really provide anything substantial, anyway. However, Python being the language it is, an appropriately behaving class is quite easily written from scratch. Except that if you write your own class from scratch, you can't use it as a slice. For a language that is supposed to be about duck typing I find it strange that if I make my own class with a start, stop and step attribute, that python barfs on it when I want to use it as a slice. class sl(object): ... def __init__(self, start = None, stop = None, step = None): ... self.start = start ... self.stop = stop ... self.step = step ... lst = range(20) s1 = slice(3,13) s2 = sl(3,13) s1.start 3 s2.start 3 s1.stop 13 s2.stop 13 s1.step s2.step lst[s1] [3, 4, 5, 6, 7, 8, 9, 10, 11, 12] lst[s2] Traceback (most recent call last): File stdin, line 1, in ? TypeError: list indices must be integers -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Antoon Pardon wrote: [Subclasses of list or slice for ranges] Except that if you write your own class from scratch, you can't use it as a slice. For a language that is supposed to be about duck typing I find it strange that if I make my own class with a start, stop and step attribute, that python barfs on it when I want to use it as a slice. [s2 has start, stop, step...] lst[s2] Traceback (most recent call last): File stdin, line 1, in ? TypeError: list indices must be integers In fact, the duck typing seems only to really work outside the interpreter and the various accompanying built-in classes. Consider a class similar to the one you defined with the name myslice (for clarity) and with the apparently necessary indices method; consider it used as follows: range(0, 10)[myslice(1, 5, 2)] Traceback (most recent call last): File stdin, line 1, in ? TypeError: list indices must be integers Here, we start to believe that only traditional index or slice notation will work. However, we did come across the built-in slice class before; consider this: range(0, 10)[slice(1, 5, 2)] [1, 3] Regardless of whether myslice inherits from object or not, there's no persuading the interpreter that it is a genuine slice, and remember that we can't subclass slice (for some reason unknown). So, it would appear that the interpreter really wants instances from some specific set of types (presumably discoverable by looking at list_subscript in listobject.c) rather than some objects conforming to some interface or protocol, and perhaps it is implemented this way for performance reasons. In any case, in the core of Python some types/classes are more equal than others, and for whatever reason the duck typing breaks down - a case of malbik endar [1] that you just have to be aware of, I suppose. Paul [1] http://www.sciamanna.com/island/pop/2004_07_13/280_malbik_endar.htm -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Paul Boddie wrote: [...] Regardless of whether myslice inherits from object or not, there's no persuading the interpreter that it is a genuine slice, and remember that we can't subclass slice (for some reason unknown). So, it would appear that the interpreter really wants instances from some specific set of types (presumably discoverable by looking at list_subscript in listobject.c) rather than some objects conforming to some interface or protocol, and perhaps it is implemented this way for performance reasons. In any case, in the core of Python some types/classes are more equal than others, and for whatever reason the duck typing breaks down - a case of malbik endar [1] that you just have to be aware of, I suppose. class mySlice(types.SliceType): ... pass ... Traceback (most recent call last): File stdin, line 1, in module TypeError: Error when calling the metaclass bases type 'slice' is not an acceptable base type Indeed. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Paul Boddie [EMAIL PROTECTED] wrote: John Machin wrote: range() and xrange() are functions. You are suggesting that 2 *functions* should acquire a __contains__ method each? I trust not. Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). You'd also have to override just about every mutating method to switch back to a normal __contains__ (or change self's type on the fly) -- a pretty heavy price to pay. I have often noticed that subclassing list, dict and maybe set has this kind of issue: the need to track every possible change to the object. Maybe a good mechanism to have for the purpose would be to add to mutable types a hook method, say __mutator__, which gets called either right before or right after any mutating method (there are different tradeoffs for before-calls and after-calls), presumably passing along the *a and **k for generality (although it might be faster for the base case to avoid that); the base types would have a no-op implementation, but subtypes could easily override just the hook to facilitate their task of maintaining extra state (could be as little as a per-instance flag recording whether the object is guaranteed to be still pristine). At C level, that might be an extra slot tp_mutator, left NULL in base types to indicate no mutator-hook method implemented here. Like any other addition of, or change to, functionality, this would of course be a proposal for 2.6, since 2.5 is feature-frozen now. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Alex Martelli wrote: Paul Boddie [EMAIL PROTECTED] wrote: Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). You'd also have to override just about every mutating method to switch back to a normal __contains__ (or change self's type on the fly) -- a pretty heavy price to pay. A subclass of list is probably a bad idea in hindsight, due to various probable requirements of it actually needing to be a list with all its contents, whereas we wanted to avoid having anything like a list around until the contents of this lazy list were required by the program. If we really wanted to subclass something, we could consider subclassing the slice class/type, but that isn't subclassable in today's Python for some reason, and it doesn't really provide anything substantial, anyway. However, Python being the language it is, an appropriately behaving class is quite easily written from scratch. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Paul Boddie [EMAIL PROTECTED] wrote: Alex Martelli wrote: Paul Boddie [EMAIL PROTECTED] wrote: Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). You'd also have to override just about every mutating method to switch back to a normal __contains__ (or change self's type on the fly) -- a pretty heavy price to pay. A subclass of list is probably a bad idea in hindsight, due to various probable requirements of it actually needing to be a list with all its contents, whereas we wanted to avoid having anything like a list around until the contents of this lazy list were required by the program. If we really wanted to subclass something, we could consider subclassing the slice class/type, but that isn't subclassable in today's Python for some reason, and it doesn't really provide anything substantial, anyway. However, Python being the language it is, an appropriately behaving class is quite easily written from scratch. Nevertheless, that class will still need to implement every single method of the list type; making it a subclass of list has some advantage in that every such implementation of a method can basically fill the real list, self.__class__=list, and leave all the rest, forevermore (explicitly here, implicitly in the future), to class list. Performance should be much better than by working off semi-deprecated UserList. A hook method __mutator__ (ideally called _before_ in this case), as I was proposing (for 2.6 or later), would make such approaches way easier and handier (and would help with most use cases I can think of for subclassing list, dict or set). Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Paul Boddie wrote: John Machin wrote: On 19/07/2006 1:05 AM, Dan Bishop wrote: xrange already has __contains__. As pointed out previously, xrange is a function and one would not expect it to have a __contains__ method. Well, you pointed out that range is a function, but xrange seems to be a type... xrange type 'xrange' dir(xrange) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__iter__', '__len__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__str__'] No __contains__ method, though, at least in 2.4.1. The objects returned by xrange do not (according to my reading of the 2.4.3 version of Objects/rangeobject.c) have a __contains__ method. As confirmed by the above evidence. I find it difficult to believe that an inefficient __contains__ has been implemented since. So do I. As you go on to say, the usual sequence traversal mechanisms are probably used to support the in operator. Whether it's a pressing matter to add support for a more efficient mechanism depends on how often people want to use ranges in the way described. Perhaps I'll write a patch - who knows? ;-) My mistake. I should have looked at dir(xrange) before posting. But the point remains that xrange's implicit __contains__ runs in linear time when a constant-time algorithm exists. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
In [EMAIL PROTECTED], tac-tics wrote: Grant Edwards wrote: for pete's sake use the comparison operator like god intended. if 0 = i = 1: I'm assuming you used Python's compound comparison as opposed to the C-style of and'ing two comparisons together to emphasize the fact it is god's chosen way of doing this ;-) Pete doesn't like to be called god in public. ;-) Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Grant Edwards [EMAIL PROTECTED] wrote: Creating and then searching a 10,000 element list to see if a number is between two other numbers is insane. Using xrange as somebody else suggested is also insane. Aye to both If you want to know if a number is between two other numders, for pete's sake use the comparison operator like god intended. if 0 = i = 1: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like $ python2.4 -m timeit -s 's=range(0,1); i=5000' 'i in s' 1000 loops, best of 3: 228 usec per loop $ python2.4 -m timeit -s 's=set(range(0,1)); i=5000' 'i in s' 100 loops, best of 3: 0.312 usec per loop $ python2.4 -m timeit -s 'i=5000' '0 = i 1' 100 loops, best of 3: 0.289 usec per loop The below prints range) That took 21.512 seconds: result 10001.0 set) That took 0.023 seconds: result 10001.0 comparison) That took 0.024 seconds: result 10001.0 import time start = time.time() a = 1.0 for i in range(0, 3): if i in range(0, 1): a += 1 dt = time.time() - start print range) That took %.3f seconds: result %s % (dt, a) start = time.time() a = 1.0 mine = set(range(0, 1)) for i in range(0, 3): if i in mine: a += 1 dt = time.time() - start print set) That took %.3f seconds: result %s % (dt, a) start = time.time() a = 1.0 mine = set(range(0, 1)) for i in range(0, 3): if 0 = i 1: a += 1 dt = time.time() - start print comparison) That took %.3f seconds: result %s % (dt, a) -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
John Machin wrote: On 18/07/2006 12:41 PM, [EMAIL PROTECTED] wrote: it seems that range() can be really slow: [...] Some things to try: 1a. Read what the manual has to say about the range() function ... what does it produce? Indeed. Still, the addition of a __contains__ method to range (and xrange) would permit acceptable performance for the code given. Perhaps this is a convenience worth considering for future Python releases. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
[EMAIL PROTECTED] wrote: it seems that range() can be really slow: if i in range (0, 1): RTFM on range() You're completely mis-using it here, using it with an if ... in ... test. The purpose of range() in Python is as loop control, not comparisons! It's not a SQL BETWEEN statement. Although you _can_ do this (you've done it!) you've also found that it's slow. Many people would argue that even using range() for loop control is unusably slow. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Grant Edwards wrote: Using xrange as somebody else suggested is also insane. Sorry about that, I somehow got the misguided notion that xrange defines its own __contains__, so that it would be about the same speed as using comparison operators directly. I figured the OP might have a better reason for wanting to use range() than his post mentioned -- perhaps the range to check was being passed from a function, and it would be easier to pass an object than a tuple of lower and upper bound -- but since xrange does looping for a membership test, my suggestion was indeed insane. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 18/07/2006 7:22 PM, Paul Boddie wrote: John Machin wrote: On 18/07/2006 12:41 PM, [EMAIL PROTECTED] wrote: it seems that range() can be really slow: [...] Some things to try: 1a. Read what the manual has to say about the range() function ... what does it produce? Indeed. Still, the addition of a __contains__ method to range (and xrange) would permit acceptable performance for the code given. Perhaps this is a convenience worth considering for future Python releases. range() and xrange() are functions. You are suggesting that 2 *functions* should acquire a __contains__ method each? I trust not. Perhaps you meant that the acquisitors should be the objects that those functions return. range() returns a list. Lists already have a __contains__ method. It's been getting a fair old exercising in the last few hours, but here we go again: python -mtimeit -sx=range(3) x.__contains__(4) 1000 loops, best of 3: 1.09 msec per loop python -mtimeit -sx=range(3) 4 in x 1000 loops, best of 3: 1.09 msec per loop python -mtimeit -sx=range(3) x.__contains__(1) 100 loops, best of 3: 0.434 usec per loop python -mtimeit -sx=range(3) 1 in x 100 loops, best of 3: 0.137 usec per loop A __contains__ method for xrange objects? The case of x in xrange(lo, hi) is met by lo = x hi. IMHO, anyone who wants x in xrange(lo, hi, step) should be encouraged to do the necessary algebra themselves; I wouldn't suggest that any core dev time be wasted on implementing such folderol. In any case, isn't the *whole* xrange gizmoid on GvR's I-wish-I-hadn't list? Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
John Machin wrote: range() and xrange() are functions. You are suggesting that 2 *functions* should acquire a __contains__ method each? I trust not. Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). Perhaps you meant that the acquisitors should be the objects that those functions return. The whole point was to avoid expanding the range into a list. [...] A __contains__ method for xrange objects? The case of x in xrange(lo, hi) is met by lo = x hi. IMHO, anyone who wants x in xrange(lo, hi, step) should be encouraged to do the necessary algebra themselves; I wouldn't suggest that any core dev time be wasted on implementing such folderol. Sure, you could always implement your own class to behave like an existing range and to implement the efficient semantics proposed above. In any case, isn't the *whole* xrange gizmoid on GvR's I-wish-I-hadn't list? Yes, he wants range to return an iterator, just like xrange more or less does now. Given that xrange objects support __getitem__, unlike a lot of other iterators (and, of course, generators), adding __contains__ wouldn't be much of a hardship. Certainly, compared to other notational conveniences bounced around on the various development lists, this one would probably provide an order of magnitude improvement on the usual bang per buck development ratio. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Andy Dingley wrote: The purpose of range() in Python is as loop control, No, the purpose of range() is to create a list, as the docs state. http://docs.python.org/lib/built-in-funcs.html range(...) - This is a versatile function to create lists containing arithmetic progressions. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Paul Boddie wrote: Yes, he wants range to return an iterator, just like xrange more or less does now. Given that xrange objects support __getitem__, unlike a lot of other iterators (and, of course, generators), adding __contains__ wouldn't be much of a hardship. Certainly, compared to other notational conveniences bounced around on the various development lists, this one would probably provide an order of magnitude improvement on the usual bang per buck development ratio. xrange already has __contains__. The problem is, it's implemented by a highly-inefficient sequential search. Why not modify it to merely check the bounds and (value - start) % step == 0? -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, tac-tics [EMAIL PROTECTED] wrote: Grant Edwards wrote: for pete's sake use the comparison operator like god intended. if 0 = i = 1: I'm assuming you used Python's compound comparison as opposed to the C-style of and'ing two comparisons together to emphasize the fact it is god's chosen way of doing this ;-) Exactly! -- Grant Edwards grante Yow! HUGH BEAUMONT died at in 1982!! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: In [EMAIL PROTECTED], tac-tics wrote: Grant Edwards wrote: for pete's sake use the comparison operator like god intended. if 0 = i = 1: I'm assuming you used Python's compound comparison as opposed to the C-style of and'ing two comparisons together to emphasize the fact it is god's chosen way of doing this ;-) Pete doesn't like to be called god in public. ;-) Interesting point. Does the phrase for pete's sake as god intended equate pete with god? It's possible that pete is not god and yet god's intentions are in pete's best interest, so something could be for pete's sake and as god intended without pete being god. That said, for pete's sake is probably a just an cleaned up version of for god's sake, so I probably did call pete god. -- Grant Edwards grante Yow! This PORCUPINE knows at his ZIPCODE... And he has visi.comVISA!! -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, Paul Boddie [EMAIL PROTECTED] wrote: John Machin wrote: range() and xrange() are functions. You are suggesting that 2 *functions* should acquire a __contains__ method each? I trust not. Well, range is a function in the current implementation, although its usage is similar to that one would get if it were a class, particularly a subclass of list or one providing a list-style interface. With such a class, you could provide a __contains__ method which could answer the question of what the range contains based on the semantics guaranteed by a range (in contrast to a normal list). Perhaps you meant that the acquisitors should be the objects that those functions return. The whole point was to avoid expanding the range into a list. It's unclear what you're referring to as the range. Perhaps you're thinking of a slice? Somethign like if (0:1).contains(x): -- Grant Edwards grante Yow! Make me look like at LINDA RONSTADT again!! visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Grant Edwards wrote: On 2006-07-18, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: In [EMAIL PROTECTED], tac-tics wrote: Grant Edwards wrote: for pete's sake use the comparison operator like god intended. if 0 = i = 1: I'm assuming you used Python's compound comparison as opposed to the C-style of and'ing two comparisons together to emphasize the fact it is god's chosen way of doing this ;-) Pete doesn't like to be called god in public. ;-) Interesting point. Does the phrase for pete's sake as god intended equate pete with god? It's possible that pete is not god and yet god's intentions are in pete's best interest, so something could be for pete's sake and as god intended without pete being god. That said, for pete's sake is probably a just an cleaned up version of for god's sake, so I probably did call pete god. No, actually you called god pete ;-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Dan Bishop wrote: [EMAIL PROTECTED] wrote: it seems that range() can be really slow: ... if i in range (0, 1): This creates a 10,000-element list and sequentially searches it. Of course that's gonna be slow. And you're doing it 3 times. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Nick Craig-Wood wrote: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like I know this is self-evident to most of the people reading this, but I thought it worth pointing out that this is a great way to test membership in range(lo, hi, step) without doing the necessary algebra. i.e. n in set(xrange(0, 1, 23)) ... Peace, ~Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Simon Forman wrote: Nick Craig-Wood wrote: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like I know this is self-evident to most of the people reading this, but I thought it worth pointing out that this is a great way to test membership in range(lo, hi, step) without doing the necessary algebra. i.e. n in set(xrange(0, 1, 23)) ... No, its not. It works, but it works by no means faster than n in range(0, 1, 23) Both need O(n), which is a bit slow compared to (((n - 15) % 23) == 0 and n = 15 and n 1) that will run in O(1) Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Simon Forman wrote: Nick Craig-Wood wrote: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like I know this is self-evident to most of the people reading this, but I thought it worth pointing out that this is a great way to test membership in range(lo, hi, step) without doing the necessary algebra. i.e. n in set(xrange(0, 1, 23)) ... This is very very misleading... here are some timings : python -mtimeit n=5000 n in set(xrange(0,1)) 1000 loops, best of 3: 1.32 msec per loop python -mtimeit n=5000 n in xrange(0,1) 1000 loops, best of 3: 455 usec per loop python -mtimeit n=5000 0 = n 1 100 loops, best of 3: 0.217 usec per loop sets are fast only if you create them *once* and use them again and again. even in that case, the sets use up O(n) memory. with comparison operators, you don't need extra memory *and* there is no pre-computation required. [sreeram;] signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Diez B. Roggisch wrote: Simon Forman wrote: Nick Craig-Wood wrote: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like I know this is self-evident to most of the people reading this, but I thought it worth pointing out that this is a great way to test membership in range(lo, hi, step) without doing the necessary algebra. i.e. n in set(xrange(0, 1, 23)) ... No, its not. It works, but it works by no means faster than n in range(0, 1, 23) Both need O(n), which is a bit slow compared to (((n - 15) % 23) == 0 and n = 15 and n 1) that will run in O(1) Diez You're right, of course. I should have said that if you're testing such a membership, say, 3 times AND you (like me) are slightly rusty on the algebra and prefer to let the computer do it, then you could use something like: test_set = set(xrange(0, 1, 23)) if n in test_set: ... ;-) ~Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Grant Edwards wrote: It's unclear what you're referring to as the range. The notion of something describing a range of values which can be expanded to a list or, of relevance here, whose boundaries can be tested efficiently. Perhaps you're thinking of a slice? Somethign like if (0:1).contains(x): Did you mean...? (0:1) # SyntaxError slice(0, 1).contains(x) # AttributeError 3 in slice(0, 1) # TypeError Something like this might suffice if slice could have a __contains__ method or if people thought of slices as natural things to test against. Perhaps we could ask the original questioner why they chose to use range in such a way - it might indicate a background in languages which encourage the construction of ranges and their use in comparisons - although being told to RTFM may have scared them off. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, Steve Holden [EMAIL PROTECTED] wrote:
That said, for pete's sake is probably a just an cleaned up
version of for god's sake, so I probably did call pete god.
No, actually you called god pete ;-)
Well that's certainly wrong, because we all know god's name is
Howard.
Our father who art in heaven,
Howard be thy name,
--
Grant Edwards grante Yow! .. I have a
at VISION! It's a RANCID
visi.comdouble-FISHWICH on an
ENRICHED BUN!!
--
http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, Paul Boddie [EMAIL PROTECTED] wrote: It's unclear what you're referring to as the range. The notion of something describing a range of values which can be expanded to a list or, of relevance here, whose boundaries can be tested efficiently. Perhaps you're thinking of a slice? Somethign like if (0:1).contains(x): I didn't mean to imply that would actually work, but I thought maybe that's what you were proposing. Did you mean...? (0:1) # SyntaxError slice(0, 1).contains(x) # AttributeError 3 in slice(0, 1) # TypeError Something like this might suffice if slice could have a __contains__ method or if people thought of slices as natural things to test against. A slice seems to me to be the obvious way to represent a finite length algebraic sequence of integers. However, obvioiusness is in the eye of the beholder since as you point out below to the OP, a range() was the obvious way to it. Perhaps we could ask the original questioner why they chose to use range in such a way - it might indicate a background in languages which encourage the construction of ranges and their use in comparisons - although being told to RTFM may have scared them off. -- Grant Edwards grante Yow! Did I say I was a at sardine? Or a bus??? visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Dan Bishop: xrange already has __contains__. The problem is, it's implemented by a highly-inefficient sequential search. Why not modify it to merely check the bounds and (value - start) % step == 0? I think this is a nice idea. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
K.S.Sreeram wrote: Simon Forman wrote: Nick Craig-Wood wrote: Sets are pretty fast too, and have the advantage of flexibility in that you can put any numbers in you like I know this is self-evident to most of the people reading this, but I thought it worth pointing out that this is a great way to test membership in range(lo, hi, step) without doing the necessary algebra. i.e. n in set(xrange(0, 1, 23)) ... This is very very misleading... here are some timings : Yes it is. I'm sorry about that. python -mtimeit n=5000 n in set(xrange(0,1)) 1000 loops, best of 3: 1.32 msec per loop python -mtimeit n=5000 n in xrange(0,1) 1000 loops, best of 3: 455 usec per loop python -mtimeit n=5000 0 = n 1 100 loops, best of 3: 0.217 usec per loop sets are fast only if you create them *once* and use them again and again. even in that case, the sets use up O(n) memory. That's what I meant. But I didn't state it clearly. One of the things I like most about python is that it allows you to specify the problem that you want to solve without a great deal of difficulty as to *how* to specify it. To me, and perhaps others, T = set(xrange(0, 1, 23)) and n in T are somewhat easier to read and write than not n % 23 and 0 = n 1, YMMV. In the given case a set of ~(1 / 23) ints would not usually be too burdensome on ram, and the code runs close to the same speed as compared to the direct calculation: from timeit import Timer times = 10 Max = 1 n = 5000 T = set(xrange(0, Max, 23)) s1 = 'n in T' s2 = 'not n %% 23 and 0 = n %s' % Max setup = 'from __main__ import n, T' S1 = Timer(s1, setup).repeat(number=times) S2 = Timer(s2, setup).repeat(number=times) print %.3f usec/pass % (100 * min(S1) / times) print %.3f usec/pass % (100 * min(S2) / times) On my machine this printed: 0.476 usec/pass 0.552 usec/pass with comparison operators, you don't need extra memory *and* there is no pre-computation required. When I set Max = 1 in the above test code there was serious disk thrashing... ;-) [sreeram;] FWIW, in production code I would certainly use the comparison operators. A kilobyte saved is a kilobyte earned. Peace, ~Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
[EMAIL PROTECTED] wrote: it seems that range() can be really slow: if i in range (0, 1): My original use was like this: if i in range (iStart, iEnd): listData.append(a) in which iStart is 1000 and iEnd is 1008 so in that case, the program ran fine... but later on, i wanted to include all data, so I relaxed the range by setting iStart to 0 and iEnd to and later on i found that the program was slow due to this. So looks like the usage of if sDay in (Tue, Wed, Thu): is more like good use of in a list but in range(0,1) will be a big search in a list. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Simon Forman wrote: To me, and perhaps others, T = set(xrange(0, 1, 23)) and n in T are somewhat easier to read and write than not n % 23 and 0 = n 1, YMMV. Eh? How is the first easier to read than the second?? You have a nested function call in the first! Regardless, testing if a member is part of a ranged set is always going to be slower. It's the nature of what you're doing. Building a set and then searching it takes much longer than a single modulus and subtraction (which is all an integer comparison is). -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 19/07/2006 1:05 AM, Dan Bishop wrote: Paul Boddie wrote: Yes, he wants range to return an iterator, just like xrange more or less does now. Given that xrange objects support __getitem__, unlike a lot of other iterators (and, of course, generators), adding __contains__ wouldn't be much of a hardship. Certainly, compared to other notational conveniences bounced around on the various development lists, this one would probably provide an order of magnitude improvement on the usual bang per buck development ratio. xrange already has __contains__. As pointed out previously, xrange is a function and one would not expect it to have a __contains__ method. The objects returned by xrange do not (according to my reading of the 2.4.3 version of Objects/rangeobject.c) have a __contains__ method. I find it difficult to believe that an inefficient __contains__ has been implemented since. Perhaps you are unaware that the mere fact that an object supports the in operation does not mean that this support is provided by a __contains__ method. The following section of the manual may help: The membership test operators (in and not in) are normally implemented as an iteration through a sequence. However, container objects can supply the following special method with a more efficient implementation, which also does not require the object be a sequence. __contains__( self, item) Called to implement membership test operators. Should return true if item is in self, false otherwise. For mapping objects, this should consider the keys of the mapping rather than the values or the key-item pairs. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
John Machin wrote: On 19/07/2006 1:05 AM, Dan Bishop wrote: xrange already has __contains__. As pointed out previously, xrange is a function and one would not expect it to have a __contains__ method. Well, you pointed out that range is a function, but xrange seems to be a type... xrange type 'xrange' dir(xrange) ['__class__', '__delattr__', '__doc__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__iter__', '__len__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__str__'] No __contains__ method, though, at least in 2.4.1. The objects returned by xrange do not (according to my reading of the 2.4.3 version of Objects/rangeobject.c) have a __contains__ method. As confirmed by the above evidence. I find it difficult to believe that an inefficient __contains__ has been implemented since. So do I. As you go on to say, the usual sequence traversal mechanisms are probably used to support the in operator. Whether it's a pressing matter to add support for a more efficient mechanism depends on how often people want to use ranges in the way described. Perhaps I'll write a patch - who knows? ;-) Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
tac-tics wrote: Simon Forman wrote: To me, and perhaps others, T = set(xrange(0, 1, 23)) and n in T are somewhat easier to read and write than not n % 23 and 0 = n 1, YMMV. Eh? How is the first easier to read than the second?? You have a nested function call in the first! I find the first form more immediately comprehensible than the latter. I know what xrange() does, and I know what set() does, and nested function calls give me no trouble, whereas the latter form with a modulus, negation, and comparisons would take me a bit longer both to compose and/or understand. If this is not the case for you then by all means please disregard my posting. YMMV. Regardless, testing if a member is part of a ranged set is always going to be slower. Yes. Roughly 0.001 seconds slower on my five year old computer. I'm not worried. It's the nature of what you're doing. Building a set and then searching it takes much longer than a single modulus and subtraction (which is all an integer comparison is). Building the set, yes, but searching the set is very close to the same speed, even for rather large sets. If you were performing the search 3 times (like in the OP) it would only take about three thousandths of a second longer, and that's on my old slow computer. If I were doing this a thousand times more often, or on a range of a million or more, or in production code, or with ranges that changed often, then I would certainly take the time to write out the latter form. Peace, ~Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
[EMAIL PROTECTED] wrote: it seems that range() can be really slow: ... if i in range (0, 1): This creates a 10,000-element list and sequentially searches it. Of course that's gonna be slow. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
[EMAIL PROTECTED] wrote: or is there an alternative use of range() or something similar that can be as fast? You could use xrange: [EMAIL PROTECTED]:~$ python -m timeit -n1 1 in range(1) 1 loops, best of 3: 260 usec per loop [EMAIL PROTECTED]:~$ python -m timeit -n1 1 in xrange(1) 1 loops, best of 3: 0.664 usec per loop -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Leif K-Brooks wrote: [EMAIL PROTECTED] wrote: or is there an alternative use of range() or something similar that can be as fast? You could use xrange: [EMAIL PROTECTED]:~$ python -m timeit -n1 1 in range(1) 1 loops, best of 3: 260 usec per loop [EMAIL PROTECTED]:~$ python -m timeit -n1 1 in xrange(1) 1 loops, best of 3: 0.664 usec per loop That's only because you're choosing a number that's early in the list. ~$ python -m timeit -n1 1 in xrange(1) 1 loops, best of 3: 1.22 usec per loop ~$ python -m timeit -n1 in xrange(1) 1 loops, best of 3: 1.24 msec per loop That's *milliseconds*, not microseconds. -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 18/07/2006 12:41 PM, [EMAIL PROTECTED] wrote: it seems that range() can be really slow: the following program will run, and the last line shows how long it ran for: import time startTime = time.time() a = 1.0 for i in range(0, 3): if i in range (0, 1): a += 1 if not i % 1000: print i print a,, round(time.time() - startTime, 1), seconds - the last line of output is - 10001.0 22.8 seconds so if i change the line if i in range (0, 1): to if i = 0 and i 1: the the last line is 10001.0 0.2 seconds so approximately, the program ran 100 times faster! or is there an alternative use of range() or something similar that can be as fast? Some things to try: 1a. Read what the manual has to say about the range() function ... what does it produce? 1b. Read what the manual has to say about time.time() and time.clock(). Change over to using time.clock(). Change the round(, 1) to (say) 4. Alternatively, use something like this: print %.1f ... %.4f seconds % (a, time.clock() - startTime) 1c. Repeat the two ways that you tried already. 2. First alternative: Do this: test_range = range(1) *once*, just after a = 1.0. and change your if test to if i in test_range: 3. Now change that to: test_range = set(range(1)) 4. Now forget about test_range, and change your if test to this: if 0 = i 1: HTH, John -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
[EMAIL PROTECTED] wrote: so if i change the line if i in range (0, 1): to if i = 0 and i 1: [snip;] is there an alternative use of range() or something similar that can be as fast? you've found that alternative yourself! just use the comparison operators... in fact, you can write a little more compact as: if 0 = i 1 : [sreeram;] signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
On 2006-07-18, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: it seems that range() can be really slow: the following program will run, and the last line shows how long it ran for: import time startTime = time.time() a = 1.0 for i in range(0, 3): if i in range (0, 1): a += 1 if not i % 1000: print i print a,, round(time.time() - startTime, 1), seconds or is there an alternative use of range() or something similar that can be as fast? Creating and then searching a 10,000 element list to see if a number is between two other numbers is insane. Using xrange as somebody else suggested is also insane. If you want to know if a number is between two other numders, for pete's sake use the comparison operator like god intended. if 0 = i = 1: -- Grant Edwards grante Yow! ANN JILLIAN'S HAIR at makes LONI ANDERSON'S visi.comHAIR look like RICARDO MONTALBAN'S HAIR! -- http://mail.python.org/mailman/listinfo/python-list
Re: range() is not the best way to check range?
Grant Edwards wrote: for pete's sake use the comparison operator like god intended. if 0 = i = 1: I'm assuming you used Python's compound comparison as opposed to the C-style of and'ing two comparisons together to emphasize the fact it is god's chosen way of doing this ;-) -- http://mail.python.org/mailman/listinfo/python-list
