Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: The issue that really bothers me here is bloating the builtin space. We already have an uncomfortable amount of builtin functions. Maybe what we're really after here is the notion of a builtin module that's pre-imported into the builtin namespace. -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Steven Bethard wrote: I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: Indeed, I see iter() as being more like len(), which is clearly a function, not a constructor. Making iter() a type and giving it class methods would be strange. -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: A minimal set would not be that offensive, yes. But then we would have two places to look for itertools functionality, which may not be desirable. True, though this is currently necessary with str objects if you want to use, say string.maketrans, so it's not without some precedent. If it's necessary to leave anything in itertools, my suggestion would be that the documentation for the iter "type" have a clear "see also" link to the itertools module. One thing that might be worth keeping in mind is that some of itertools functionality is going to become obsolete come py3k (izip->zip), and some is already (imap). At least such operations should not be dumped into the builtin iter. Yeah, maps and filters are basically obsolete as of generator expressions. The list of itertools functions that don't seem obsolete (and won't be made obsolete by Python 3.0): chain count cycle dropwhile groupby islice repeat takewhile tee As I suggested, I think that chain, islice and tee are tightly coupled with iterator objects, providing concatenation, slicing and copying operations. This leaves: count cycle dropwhile groupby repeat takewhile None of these really have analogs in sequence objects, so I consider them less tightly tied to iter. I'd probahbly say that these are more along the lines of alternate constructors, ala dict.fromkeys. While they're certainly useful at times, I'd be happy enough to leave them in itertools if that was the general feeling. Of course I guess I'd be happy enough to leave everything in itertools if that was the general feeling (or the BDFL pronouncement). ;) STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
> "Steven" == Steven Bethard <[EMAIL PROTECTED]> writes: Steven> Certainly a valid point. How would you feel about adding Steven> just a select few itertools functions, perhaps just Steven> islice, chain and tee? A minimal set would not be that offensive, yes. But then we would have two places to look for itertools functionality, which may not be desirable. One thing that might be worth keeping in mind is that some of itertools functionality is going to become obsolete come py3k (izip->zip), and some is already (imap). At least such operations should not be dumped into the builtin iter. -- Ville Vainio http://tinyurl.com/2prnb -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: The issue that really bothers me here is bloating the builtin space. We already have an uncomfortable amount of builtin functions. Of course the additions that have been suggested would not pollute the builtin namespace, but they would still be there, taking space. I'd rather see a more modular and 'slimmer' Python, what with the advent of Python for S60 and other embedded uses. Certainly a valid point. How would you feel about adding just a select few itertools functions, perhaps just islice, chain and tee? These functions provide the operations that exist for lists but don't, by default, exist for iterators: slicing, concatenation and copying. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
> "Steven" == Steven Bethard <[EMAIL PROTECTED]> writes: Steven> to be documented as a builtin type. I don't find the Steven> argument "builtin type methods are hard to find" Steven> convincing -- the solution here is to fix the Steven> documentation, not refuse to add builtin types. Yep - that's why we should perhaps fix the documentation first :-). Steven> I guess the real questions are[1]: Steven> * How much does iter feel like a type? Guess this depends on the person. I've never thought of it as a type. It's too fundamental a concept to coerce into a type, even thought protocol == type in a sense. Steven> [1] There's also the question of how much you believe in Steven> OO tenets like "functions closely associated with a type Steven> should be members of that type"... The issue that really bothers me here is bloating the builtin space. We already have an uncomfortable amount of builtin functions. Of course the additions that have been suggested would not pollute the builtin namespace, but they would still be there, taking space. I'd rather see a more modular and 'slimmer' Python, what with the advent of Python for S60 and other embedded uses. Perhaps what you need is 'from usefulstuff import *', with usefulstuff having os, sys, 'itertools as it', &c. -- Ville Vainio http://tinyurl.com/2prnb -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Terry Reedy wrote: But if classmethods are intended to provide alternate constructors But I do not remember that being given as a reason for classmethod(). But I am not sure what was. Well I haven't searched thoroughly, but I know one place that it's referenced is in descrintro[1]: "Factoid: __new__ is a static method, not a class method. I initially thought it would have to be a class method, and that's why I added the classmethod primitive. Unfortunately, with class methods, upcalls don't work right in this case, so I had to make it a static method with an explicit class as its first argument. Ironically, there are now no known uses for class methods in the Python distribution (other than in the test suite). However, class methods are still useful in other places, for example, to program inheritable alternate constructors." Not sure if this is the only reason though, and even if it is, it might not be entirely applicable because while the itertools functions may be supplying alternate constructors, it's not clear why anyone would subclass iter[2], so the constructors aren't likely to be inherited. STeVe [1] http://www.python.org/2.2.3/descrintro.html#__new__ [2] That is, in the simple case, where iter is still basically a factory function, not a type wrapper. -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
"Steven Bethard" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Terry Reedy wrote: >> "Steven Bethard" <[EMAIL PROTECTED]> wrote in message >> news:[EMAIL PROTECTED] >> >>>True it's not a huge win. But I'd argue that for the same reasons that >>>dict.fromkeys is a dict classmethod, the itertools methods could be iter >>>classmethods (or staticmethods). >> >> As near as I could tell from the doc, .fromkeys is the only dict method >> that is a classmethod (better, typemethod) rather than an instance >> method. And all list methods are instance methods. And I believe the >> same is true of all number operations (and the corresponding special >> methods). So .fromkeys seems to be an anomaly. >> >> I believe the reason for its existence is that the signature for dict() >> itself was already pretty well 'used up' and Guido preferred to add an >> alternate constructor as a method rather than further complicate the >> signature of dict() by adding a fromkeys flag to signal an alternate >> interpretation of the first and possibly the second parameter. > > True enough, and I also agree with George Sakkis's sentiment that > fromkeys() isn't really necessary now that set() is a builtin. So perhaps it will disappear in the future. > But if classmethods are intended to provide alternate constructors But I do not remember that being given as a reason for classmethod(). But I am not sure what was. Terry J. Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Terry Reedy wrote: "Steven Bethard" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] True it's not a huge win. But I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). As near as I could tell from the doc, .fromkeys is the only dict method that is a classmethod (better, typemethod) rather than an instance method. And all list methods are instance methods. And I believe the same is true of all number operations (and the corresponding special methods). So .fromkeys seems to be an anomaly. I believe the reason for its existence is that the signature for dict() itself was already pretty well 'used up' and Guido preferred to add an alternate constructor as a method rather than further complicate the signature of dict() by adding a fromkeys flag to signal an alternate interpretation of the first and possibly the second parameter. True enough, and I also agree with George Sakkis's sentiment that fromkeys() isn't really necessary now that set() is a builtin. But if classmethods are intended to provide alternate constructors then one could argue that the functions in itertools are appropriate as they all produce iterators and are thus something like alternate iter constructors. Of course you don't want every function that produces an iterator as a member of the iter type, just like you don't want every function that produces a dict as a member of the dict type. But I could see that it might be reasonable to put some of the more commonly used "alternate constructors" there... STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
"Steven Bethard" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > True it's not a huge win. But I'd argue that for the same reasons that > dict.fromkeys is a dict classmethod, the itertools methods could be iter > classmethods (or staticmethods). As near as I could tell from the doc, .fromkeys is the only dict method that is a classmethod (better, typemethod) rather than an instance method. And all list methods are instance methods. And I believe the same is true of all number operations (and the corresponding special methods). So .fromkeys seems to be an anomaly. I believe the reason for its existence is that the signature for dict() itself was already pretty well 'used up' and Guido preferred to add an alternate constructor as a method rather than further complicate the signature of dict() by adding a fromkeys flag to signal an alternate interpretation of the first and possibly the second parameter. Terry J. Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Tue, 29 Mar 2005 11:32:33 -0800, Michael Spencer <[EMAIL PROTECTED]> wrote:
[...]
>While we're on the topic, what do you think of having unary, non-summary
>builtins automatically map themselves when called with an iterable that would
>otherwise be an illegal argument:
That last "otherwise" is pretty important for strings as in int('1234') ;-)
>
>e.g.,
>int(iterable) -> (int(i) for i in iterable)
>ord(iterable) -> (ord(i) for i in iterable)
>
>
>This would be unambiguous, I think, in the cases of bool, int, callable, chr,
>float, hex, id, long, oct, ord, vars...
>
>It would shorten the common cases of:
>for char in somestring:
> ordchar = ord(char)
> # do something with ordchar, but not char
But wouldn't you really currently write the "->" form from above? I.e.,
for ordchar in (ord(c) for c in somestring):
...
to compare with
>to
>for ordchar in ord(somestring):
> ...
>
So it's not _that_ much shorter ;-)
>It would not work for summarizing functions or those that can accept an
>iterable
>today e.g., len, repr
>
I like concise expression, so I'm willing to try it. I guess it would be enough
to override __builtins__ to get a taste, e.g., (not thought through):
>>> class itint(int):
... oldint = __builtins__.int
... def __new__(cls, arg):
... try: return cls.oldint(arg)
... except (TypeError, ValueError):
... oi = cls.oldint
... return (oi(item) for item in arg)
...
>>> __builtins__.int = itint
>>> int('1234')
1234
>>> for x in int('1 23 456'.split()): print x,
...
1 23 456
>>> for x in int(range(1,8,2)): print x,
...
1 3 5 7
>>> for x in int('123x'): print x,
...
1 2 3
Traceback (most recent call last):
File "", line 1, in ?
File "", line 7, in
ValueError: invalid literal for int(): x
Hm, ... ;-)
Regards,
Bengt Richter
--
http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Tue, Mar 29, 2005 at 12:38:42PM +0300, Ville Vainio wrote: > > "Raymond" == Raymond Hettinger <[EMAIL PROTECTED]> writes: > > Raymond> If the experience works out, then all you're left with is > Raymond> the trivial matter of convincing Guido that function > Raymond> attributes are a sure cure for the burden of typing > Raymond> import statements. > > For one thing, it would make it harder to find the functions from the > docs. It's easy to find the doc for 'itertools', but iter object > methods would require browsing that infamous Chapter 2 of the > documentation... > > Apart from that, I don't really see the advantage in moving away from > itertools. I only use itertools when I have to currently, which isn't necessarily bad (premature optimization etc) but I do use lists when I just need an iterator - simply because 'list()' is easier to type than '^^n^nimport itertools as it^x^x' (emacsen to mark HERE, jump to the top, import itertools, and jump back). If itertools methods were handier I'd use them when I just want to iterate. As an anecdote I use generator comprehensions[1] more often than list comprehensions. I'll give the builtin manipulations a try but since I have to deal with many machines I can't promise to flex it much. -jack [1] aside, I didn't care too much about upgrading machines 2.2 => 2.3, but when 2.4 came along with set as a builtin and generator comprehensions it was compelling. -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
"Steven Bethard" <[EMAIL PROTECTED]> wrote: > > [snip] > > I guess the real questions are[1]: > * How much does iter feel like a type? > * How closely are the itertools functions associated with iter? > > STeVe > > [1] There's also the question of how much you believe in OO tenets like > "functions closely associated with a type should be members of that type"... I would answer positively for both: iter does feel like a type conceptually and (most, if not all) itertools would be suitable methods for such a type. Here I am referring to 'type' more as an interface (or protocol; i'm not sure of the difference) rather than a concrete class, so whether the result of iter is an iterator or a generator object is of little importance as long as it works as expected (that it, whether it makes calls to next() or __getitem__() becomes a hidden implementation detail). If iter was a type, it would also be neat to replace some itertool callables with special methods, as it has been mentioned in another thread (http://tinyurl.com/6mmmf), so that: iter(x)[a:b:c] := itertools.islice(iter(x),a,b,c) iter(x) + iter(y) := itertools.chain(iter(x), iter(y)) iter(x) * 3 := itertools.chain(* itertools.tee(iter(x), 3)) George -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Steven Bethard wrote: Ville Vainio wrote: "Raymond" == Raymond Hettinger <[EMAIL PROTECTED]> writes: Raymond> If the experience works out, then all you're left with is Raymond> the trivial matter of convincing Guido that function Raymond> attributes are a sure cure for the burden of typing Raymond> import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Well, it would only make them as hard to find as, say, dict.fromkeys, which is probably the best parallel here. Of course iter would have to be documented as a builtin type. I don't find the argument "builtin type methods are hard to find" convincing -- the solution here is to fix the documentation, not refuse to add builtin types. Apart from that, I don't really see the advantage in moving away from itertools. True it's not a huge win. But I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: py> class C(object): ... def __iter__(self): ... yield 1 ... py> iter(C()) But note that iter does produce 'iterator' objects for the old __getitem__ protocol: py> class C(object): ... def __getitem__(self, index): ... if index > 5: ... raise IndexError ... return index ... py> iter(C()) I guess the real questions are[1]: * How much does iter feel like a type? * How closely are the itertools functions associated with iter? STeVe [1] There's also the question of how much you believe in OO tenets like "functions closely associated with a type should be members of that type"... While we're on the topic, what do you think of having unary, non-summary builtins automatically map themselves when called with an iterable that would otherwise be an illegal argument: e.g., int(iterable) -> (int(i) for i in iterable) ord(iterable) -> (ord(i) for i in iterable) This would be unambiguous, I think, in the cases of bool, int, callable, chr, float, hex, id, long, oct, ord, vars... It would shorten the common cases of: for char in somestring: ordchar = ord(char) # do something with ordchar, but not char to for ordchar in ord(somestring): ... It would not work for summarizing functions or those that can accept an iterable today e.g., len, repr Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: "Raymond" == Raymond Hettinger <[EMAIL PROTECTED]> writes: Raymond> If the experience works out, then all you're left with is Raymond> the trivial matter of convincing Guido that function Raymond> attributes are a sure cure for the burden of typing Raymond> import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Well, it would only make them as hard to find as, say, dict.fromkeys, which is probably the best parallel here. Of course iter would have to be documented as a builtin type. I don't find the argument "builtin type methods are hard to find" convincing -- the solution here is to fix the documentation, not refuse to add builtin types. Apart from that, I don't really see the advantage in moving away from itertools. True it's not a huge win. But I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: py> class C(object): ... def __iter__(self): ... yield 1 ... py> iter(C()) But note that iter does produce 'iterator' objects for the old __getitem__ protocol: py> class C(object): ... def __getitem__(self, index): ... if index > 5: ... raise IndexError ... return index ... py> iter(C()) I guess the real questions are[1]: * How much does iter feel like a type? * How closely are the itertools functions associated with iter? STeVe [1] There's also the question of how much you believe in OO tenets like "functions closely associated with a type should be members of that type"... -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
> "Raymond" == Raymond Hettinger <[EMAIL PROTECTED]> writes: Raymond> If the experience works out, then all you're left with is Raymond> the trivial matter of convincing Guido that function Raymond> attributes are a sure cure for the burden of typing Raymond> import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Apart from that, I don't really see the advantage in moving away from itertools. -- Ville Vainio http://tinyurl.com/2prnb -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
[Jack Diederich]
> > itertools to iter transition, huh? I slipped that one in, I mentioned
> > it to Raymond at PyCon and he didn't flinch. It would be nice not to
> > have to sprinkle 'import itertools as it' in code. iter could also
> > become a type wrapper instead of a function, so an iter instance could
> > be a wrapper that figures out whether to call .next or __getitem__
> > depending on it's argument.
> > for item in iter(mylist).imap:
> > print item
> > or
> > for item in iter.imap(mylist):
> > print item
[Steven Bethard]
> Very cool idea. I think the transition from
> itertools.XXX(iterable, *args, **kwargs)
> to
> iter.XXX(iterable, *args, **kwargs)
> ought to be pretty easy.
Just to make sure you guys can live with your proposed syntax, trying using it
for a month or so and report back on whether the experience was pleasant. Try
dropping the following into your setup.py
def wrapiter():
import __builtin__, itertools
orig = __builtin__.iter
def iter(*args):
return orig(*args)
for name in ('__doc__', '__name__'):
setattr(iter, name, getattr(orig, name))
vars(iter).update(vars(itertools))
__builtin__.iter = iter
wrapiter()
If the experience works out, then all you're left with is the trivial matter of
convincing Guido that function attributes are a sure cure for the burden of
typing import statements.
Raymond Hettinger
--
http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
In article <[EMAIL PROTECTED]>, Jack Diederich <[EMAIL PROTECTED]> wrote: > I only included making iter a type to make it more symmetric with str > being a type. iter is currently a function, as a practical matter I wouldn't > mind if it doubled as a namespace but that might make others flinch. iter having the attributes currently residing as methods in itertools sounds just fine to me. I really don't like iter as a type instead of a function, though. It sounds like a cool idea at first glance, but then you think about it and realize that (unlike what happens with any class name) iter(x) is almost never going to return an object of that type. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Mon, Mar 28, 2005 at 10:28:29AM -0700, Steven Bethard wrote: > Jack Diederich wrote: > > > > itertools to iter transition, huh? I slipped that one in, I mentioned > > it to Raymond at PyCon and he didn't flinch. It would be nice not to > > have to sprinkle 'import itertools as it' in code. iter could also > > become a type wrapper instead of a function, so an iter instance could > > be a wrapper that figures out whether to call .next or __getitem__ > > depending on it's argument. > > for item in iter(mylist).imap: > > print item > > or > > for item in iter.imap(mylist): > > print item > > Very cool idea. I think the transition from > itertools.XXX(iterable, *args, **kwargs) > to > iter.XXX(iterable, *args, **kwargs) > ought to be pretty easy. The transition from here to > iter(iterable).XXX(*args, **kwargs) > seems like it might be more complicated though -- iter would have to > return a proxy object instead of the object returned by __iter__[1]. I > guess it already does that for objects that support only the __getitem__ > protocol though, so maybe it's not so bad... I only included making iter a type to make it more symmetric with str being a type. iter is currently a function, as a practical matter I wouldn't mind if it doubled as a namespace but that might make others flinch. > [1] And you'd probably also want to special-case this so that if iter() > was called on an object that's already an instance of iter, that the > object itself was returned, not a proxy. -- http://mail.python.org/mailman/listinfo/python-list
