Re: list.clear() missing?!?
Ben C wrote: I used to think it assigned with things like integers, because if you write: a = 5 b = a b += 1 print a a is still 5. So it looked like a and b stored values and b got a copy of a's value. But this is the wrong interpretation, b += 1 is really b = b + 1, and rebinds b. almost: b += obj is really b = b.__iadd__(obj) with b = b + obj as a fallback if __iadd__ isn't supported by b. If it weren't for the id() function I think the difference between variable stores value, variable stores immutable reference and variable stores copy-on-write reference would be implementation detail and never visible to the programmer. except when they are: a = [1, 2, 3] b = a b += [4] a [1, 2, 3, 4] b [1, 2, 3, 4] /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Dennis Lee Bieber [EMAIL PROTECTED] writes: On Fri, 14 Apr 2006 09:17:05 +0400, Sergei Organov [EMAIL PROTECTED] declaimed the following in comp.lang.python: I, as a newcomer, don't have much trouble understanding the binding vs the assignment by themselves. What does somewhat confuse is dual role of the = operator, -- sometimes it means bind and other times it means assign, right? For me it seems that the language would be less It always means bind... But if the LHS is a mutable object, AND you have specified a component of that object, it is the component that is being rebound... lst[:] = [] is rebinding the elements inside the list lst, and not rebinding the name lst. Essentially, once you add any selector to the name (object[...]= or object.xxx=) you are going inside the object, and manipulating (rebinding) what is inside. If the name is used pure (object=), you are rebinding the /name/ to a different object. Me gets corrected, thanks. Now I need to unroll my mind somewhat back to figure out when and why I started to believe it sometimes assigns ;) -- Sergei. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On 2006-04-14, Sergei Organov [EMAIL PROTECTED] wrote: Dennis Lee Bieber [EMAIL PROTECTED] writes: It always means bind... But if the LHS is a mutable object, AND you have specified a component of that object, it is the component that is being rebound... lst[:] = [] [...] Me gets corrected, thanks. Now I need to unroll my mind somewhat back to figure out when and why I started to believe it sometimes assigns ;) I used to think it assigned with things like integers, because if you write: a = 5 b = a b += 1 print a a is still 5. So it looked like a and b stored values and b got a copy of a's value. But this is the wrong interpretation, b += 1 is really b = b + 1, and rebinds b. You can see what's really going on if you use the id() function on a and b during these operations. The other reason for the confusion is that I think in Java a variable either stores a value, in the case of numbers, or a reference in the case of objects (or a copy-on-write reference, which behaves like a value, in the case of strings). In Python it's better to think of it as always a reference, and to think in terms of immutable vs. mutable objects that are referred to. If it weren't for the id() function I think the difference between variable stores value, variable stores immutable reference and variable stores copy-on-write reference would be implementation detail and never visible to the programmer. That's why it's easy to be confused-- most of the time these interpretations are equivalent, so it doesn't matter which you work with. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond, I suspect we're not seeing eye to eye on this issue, but I do appreciate you taking the time and effort. Thank you. My comments follow. On Thu, 13 Apr 2006 09:34:46 -0700, Raymond Hettinger wrote: Both the pros and cons were quipped with abrupt perjoratives so the bullet points could be stated succinctly and with a bit of levity. Humour so often doesn't come across in text -- I didn't see the levity you intended. request is inane -- A generation of python programmers has found list clearing to be like other parts of the language that you get used to very quickly and do not prove to be a problem in practice. The request is in the same category as others which challenge api choices made 16 years ago; in particular, the decision to have compact APIs where the named methods do not duplicate functionality provided by syntax using operators and keywords. The request is less of a bug report and more a rejection of Guido's sense of design and his subsequent experience using his own language. Of course it isn't a bug report. It's a feature request. As for Guido's sense of design, in *this particular instance* I think he got it wrong. That's hardly a rejection of the man's overall design skills. In any case, as the creator of Python, Guido has never seen the language with the eyes of a Python beginner. All the more credit to him for creating a language which is simultaneously friendly to beginners and powerful for experts. But he isn't superhuman, his experience is not the only experience. (True, as BDFL, his experience has the final vote, but that's a whole different kettle of fish.) A bit of searching on Google finds this issue coming up frequently. It seems to me that there is a steady stream of Python programmers asking where's the list clear method?. Perhaps it isn't a monthly occurrence, but it is still very common. That surely suggests that, as powerful as slicing is, people aren't getting the connection between slicing and emptying a list, and even when they do get it, many dislike it. Since there is no evidence that these people are universally stupid, it suggests strongly that the connection is too subtle. Some may protest that the idiom is in the tutorial, that it is obvious once you know slicing and del, but the fact is that many people aren't finding it obvious or self-evident at all -- not even those who have read the tutorial. Here is another possible solution: have the list.clear method be defined this way: def clear(self): print Haven't you read the tutorial? Use del self[:] or self[:] = [] That at least will stop the procession of people asking how to clear a list, and Fredrik won't have to spend his time telling them to read the tutorial. (Hey, if it's good enough for quit and exit... *wink*) A list.clear method will make deleting items from a list more OO, consistent with almost everything else you do to lists, and less procedural. This is especially true if clear() takes an optional index (or two), allowing sections of the list to be cleared, not just the entire list. Don't shoot yourself in the foot here. If you want to advocate list.clear(), then you're hurting your chances by pushing for it to take an optional argument. Essentially, this amounts to an unwillingness to use the del-keyword and to duplicate its functionality with a named method. Deleting names from namespaces is conceptually distinct from deleting components of compound objects like lists, and should be handled differently: objects know their own contents, and can operate on themselves, but they don't know what namespace they live in or what name(s) they are known by. Deleting items from a list should be a case of object, operate on yourself, just like remove, append and so on, and hence should be specified by an object method, not a keyword like del. So you are right: I am uncomfortable having del do double duty to both delete names and operate on the internals of compound objects. It feels wrong to me. If insert were a keyword (as in insert list[i:j], obj) it would feel wrong to me too, and for the same reason. I haven't been using Python for sixteen years, but at six years and counting I'm not exactly a newbie. At first I accepted the del obj[:] idiom implicitly -- the whole language was new to me, and it was no more strange to me than any other Python syntax or idiom. But as I've got more familiar with Python, the sense of two distinct concepts being artificially forced into the same syntax has just gotten stronger. As a pragmatist, I'd be happy to see a bare list.clear() method; that would at least have the advantages of being easy to find with dir(list) and accessible with help(). But as a purist, I really think it needs optional arguments. Make of that what you will. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Duncan Booth wrote: Looking in the 'obvious' place in the Tutorial, section 5.1 'More on Lists' I found in the immediately following section 5.2 'The del statement': I read the tutorial 6 years ago, and don't read it regularly. What's in the tutorial is not really important, what can be easily looked up in library reference or interctive prompt is. There is a bit of elitism regarding the defense of del lst[:]. Many python programmers are casual programmers (was it Tim Berners-Lee that explained how great python is for casual users?) who know the language but don't remember the way everything is done. Of course someone that uses python 8 hours a day instantly recall lst[:], but a casual one will more probably launch an ipython prompt and do: [ipython]|1 l = [] [ipython]|2 l. l.__add__ l.__getslice__ l.__ne__ l.append l.__class__l.__gt__ l.__new__ l.count l.__contains__ l.__hash__ l.__reduce__ l.extend l.__delattr__ l.__iadd__ l.__reduce_ex__l.index l.__delitem__ l.__imul__ l.__repr__ l.insert l.__delslice__ l.__init__ l.__reversed__ l.pop l.__doc__ l.__iter__ l.__rmul__ l.remove l.__eq__ l.__le__ l.__setattr__ l.reverse l.__ge__ l.__len__ l.__setitem__ l.sort l.__getattribute__ l.__lt__ l.__setslice__ l.__getitem__ l.__mul__ l.__str__ [ipython]|2 # wtf? -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen wrote: * learning slices is basic to the language (this lesson shouldn't be skipped) And yet it doesn't appear to be in the tutorial. oh, please. slices are explained in the section on strings, and in the section on lists, and used to define the behaviour of the list methods in the second section on lists, ... I could have missed it, but I've looked in a number of the obvious places http://docs.python.org/tut/node5.html#SECTION00514 section 3.1.2 contains an example that shows to remove stuff from a list, in place. if you want a clearer example, please consider donating some of your time to the pytut wiki: http://pytut.infogami.com/ /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen wrote:
It's not even clear that extend needs two lines:
s = range(5)
more = list('abc')
s[:] = s + more
s
[0, 1, 2, 3, 4, 'a', 'b', 'c']
Okay, it's not obvious, but I don't think s[:]=[] is really any more
obvious as a way to clear the list.
Clearly .extend() needs to be removed from the language as it is an
unnecessary extension to the API using slicing
you just flunked the what Python has to do to carry out a certain operation
part of the how Python works, intermediate level certification.
/F
--
http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen wrote: * learning slices is basic to the language (this lesson shouldn't be skipped) And yet it doesn't appear to be in the tutorial. I could have missed it, but I've looked in a number of the obvious places, without actually going through it (again) from start to finish. Also, googling for slice site:docs.python.org, you have to go to the *sixth* entry before you can find the first mention of del x[:] and what it does. I think given the current docs it's possible to learn all kinds of things about slicing and still not make the non-intuitive leap that del x[slice] is actually how you spell delete contents of list in-place. Looking in the 'obvious' place in the Tutorial, section 5.1 'More on Lists' I found in the immediately following section 5.2 'The del statement': There is a way to remove an item from a list given its index instead of its value: the del statement. Unlike the pop()) method which returns a value, the del keyword is a statement and can also be used to remove slices from a list (which we did earlier by assignment of an empty list to the slice). The 'earlier showing assignment of an empty list to a slice' is a reference to section 3.1.4 'Lists': Assignment to slices is also possible, and this can even change the size of the list: # Replace some items: ... a[0:2] = [1, 12] a [1, 12, 123, 1234] # Remove some: ... a[0:2] = [] a [123, 1234] Both of these talk about ways to remove slices from a list. Perhaps the wording could be clearer to make it obvious that they can also be used to clear a list entirely (using the word 'clear' would certainly help people Googling for the answer). So maybe 'this can even change the size of the list or clear it completely' would be a good change for 3.1.4. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger wrote: Cons: - * learning slices is basic to the language (this lesson shouldn't be skipped) also, a clear method would simply clear the entire list. You still need to learn the assigning to/deleting slices technique any time you want to clear out part of a list. Every so often I still get an oh, I didn't know Python could do *that* moment, just had one now: s = range(10) s[::2] = reversed(s[::2]) s [8, 1, 6, 3, 4, 5, 2, 7, 0, 9] I've no idea when I might need it, but it just never occurred to me before that you can also assign/del non-contiguous slices. The symmetry does breaks down a bit here as assigning to an extended slice only lets you assign a sequence of the same length as the slice, so you can't delete an extended slice by assignment, only by using del. s = range(10) del s[::2] s [1, 3, 5, 7, 9] s = range(10) s[::2] = [] Traceback (most recent call last): File pyshell#19, line 1, in -toplevel- s[::2] = [] ValueError: attempt to assign sequence of size 0 to extended slice of size 5 -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Duncan Booth wrote: Peter Hansen wrote: * learning slices is basic to the language (this lesson shouldn't be skipped) And yet it doesn't appear to be in the tutorial. I could have missed Both of these talk about ways to remove slices from a list. Perhaps the wording could be clearer to make it obvious that they can also be used to clear a list entirely (using the word 'clear' would certainly help people Googling for the answer). So maybe 'this can even change the size of the list or clear it completely' would be a good change for 3.1.4. I added two examples of clearing a list to the section about slice assignment and del. Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Thu, 13 Apr 2006 07:49:11 +, Duncan Booth wrote: Raymond Hettinger wrote: Cons: - * learning slices is basic to the language (this lesson shouldn't be skipped) also, a clear method would simply clear the entire list. You still need to learn the assigning to/deleting slices technique any time you want to clear out part of a list. A clear method does not *necessarily* clear only the entire list. That's an choice that can be made. I for one would vote +1 for clear taking an optional index or slice, or two indices, to clear only a part of the list. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Thu, 13 Apr 2006 09:11:31 +0200, Fredrik Lundh wrote:
Peter Hansen wrote:
It's not even clear that extend needs two lines:
s = range(5)
more = list('abc')
s[:] = s + more
s
[0, 1, 2, 3, 4, 'a', 'b', 'c']
Okay, it's not obvious, but I don't think s[:]=[] is really any more
obvious as a way to clear the list.
Clearly .extend() needs to be removed from the language as it is an
unnecessary extension to the API using slicing
you just flunked the what Python has to do to carry out a certain operation
part of the how Python works, intermediate level certification.
So implementation details are part of the language now?
Out of curiosity, are those things that Python has to do the same for
CPython, Jython, IronPython and PyPy?
Personally, I think it is a crying shame that we're expected to be experts
on the specific internals of the Python interpreter before we're allowed
to point out that only one obvious way to do it just is not true, no
matter what the Zen says.
L = []
L.append(0)
L[:] = L + [1]
L[:] += [2]
L[len(L):] = [3]
L.__setslice__(len(L), -1, [4])
L.__setslice__(sys.maxint, sys.maxint, [5])
L += [6]
L
[0, 1, 2, 3, 4, 5, 6]
That's at least seven ways to do an append, and it is a double-standard to
declare that slice manipulation is the One and Only True obvious way
to clear a list, but slice manipulation is not obvious enough for
appending to a list.
No doubt under the hood, these seven ways are implemented differently.
They certainly differ in their obviousness, and I'm willing to bet that
practically nobody thinks that the slicing methods are more obvious than
append. Perhaps we're not Dutch. I daresay one method is better, faster,
or more efficient than the others, but remember the warning against
premature optimisation.
Whenever I see only one obvious way to do it being used as a knee-jerk
excuse for rejecting any change, my eyes roll. Nobody wants Python to
turn into Perl plus the kitchen sink, but it isn't as if Python is some
absolutely minimalist language with two objects and three functions. There
is no shortage of more than one way to do it convenience methods,
functions and even entire modules. And that's why Python is such a fun,
easy to use language: because most things in Python are just convenient.
When you want to append to a list, or insert into a list, you don't have
to muck about with slices, you call the obvious list method.
And so it should be for clearing all or part of a list.
--
Steven.
--
http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Alan Morgan wrote: Ah, but if you know your basic python then you wouldn't be looking for s.clear() in the first place; you'd just use s[:]=[] (or s=[], whichever is appropriate). One of very first things newcomers learn (I believe, though I don't know how soon the tutorial teaches it) is to use dir() on objects. The clear() method would show up there and quickly attract attention. Neither of the other techniques are likely to be discovered as quickly. (Well, okay, s=[] would be, I'm sure, but to many newcomers that might feel wrong as the way to empty out a list, but then we're back to wondering how often there's really a usecase for doing so.) Personally, it seems more *reasonable* to me, a novice python user, for s.clear() to behave like s=[] (or, perhaps, for s=[] and s[:]=[] to mean the same thing). The fact that it can't might be an argument for not having it in the first place. Then it's a good reason we had this thread, so you could learn something *crucial* to understanding Python and writing non-buggy code: name binding versus variables which occupy fixed memory locations like in some other languages. This has to be by far the most frequent area that newcomer's trip up. But that's another story... -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Duncan Booth wrote: Peter Hansen wrote: Looking in the 'obvious' place in the Tutorial, section 5.1 'More on Lists' I found in the immediately following section 5.2 'The del statement': There is a way to remove an item from a list given its index instead of its value: the del statement. Unlike the pop()) method which returns a value, the del keyword is a statement and can also be used to remove slices from a list (which we did earlier by assignment of an empty list to the slice). I saw that section too, but was scanning for any example of wiping out the whole list. As you point out, it's not mentioned. I don't think there's even an example of slicing with no arguments [:] for copying a list (i.e. on the right side of the =), and it's reasonable to assume (I originally did, as I recall) that this would be some kind of error... Both of these talk about ways to remove slices from a list. Perhaps the wording could be clearer to make it obvious that they can also be used to clear a list entirely (using the word 'clear' would certainly help people Googling for the answer). So maybe 'this can even change the size of the list or clear it completely' would be a good change for 3.1.4. This is quite true. After all, who imagines when offered a slice of cake that a slice might be the entire thing! The concept of slice in English strongly implies a *subset*, not the whole, so if we're not going to get a .clear() method, I do believe that the various uses of [:] should be much more explicitly pointed out in the docs. At least we'd have a ready place to point to in the tutorial, instead of this discussion cropping up every month. -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen wrote: One of very first things newcomers learn (I believe, though I don't know how soon the tutorial teaches it) let's see. lists are introduced on page 19, a more extensive discussion of lists is found on page 33, the del statement appears on page 34, and the dir() function is introduced on page 46. (page numbers from the current pytut PDF copy) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Alan Morgan wrote: In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: * s.clear() is more obvious in intent Serious question: Should it work more like s=[] or more like s[:]=[]. I'm assuming the latter, but the fact that there is a difference is an argument for not hiding this operation behind some syntactic sugar. It has to work more like s[:]=[] It's not easy for an object method to bind a whole different object to the first object's name. Generally only statements can affect namespaces (hence the uses of del that everyone remembers.) Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote: Convenience and obviousness are important for APIs -- that's why lists have pop, extend and remove methods. The only difference I can see between a hypothetical clear and these is that clear can be replaced with a one-liner, while the others need at least two, e.g. for extend: for item in seq: L.append(item) Both extend and append have one-line slice equivalents, except that the equivalents have to keep referring to the length of the list.. (have to keep finding the len function.) Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Fredrik Lundh wrote: Peter Hansen wrote: One of very first things newcomers learn (I believe, though I don't know how soon the tutorial teaches it) let's see. lists are introduced on page 19, a more extensive discussion of lists is found on page 33, the del statement appears on page 34, and the dir() function is introduced on page 46. You're spending a lot of time trying to convince me I'm wrong, yet until Georg did something about it just now, nowhere in those sections did it talk about [:] specifically, which is sort of the whole point. It is *not* obvious that although [x:] and [:y] and [x:y] take subsets of things, that leaving them both out is either legal or useful. Thankfully (to Georg), it now is. (Well, I haven't read it yet, but I'll take his word for it.) -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen wrote: You're spending a lot of time trying to convince me I'm wrong no, I'm just posting observable facts in response to various I'm too lazy to look this up, but I'll assume that things are this way posts. Thankfully (to Georg) it's fixed in the wiki too... talking about the wiki, are you and rurpy the same person, btw? the implied I'm going to pretend that you've never done anything to im- prove the situation insinuations are getting a bit tiresome... /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
I agree. Lists should have a clear method. But what's shocking is that it doesn't seem obvious to others. list.clear() is a whole lot more readable, intuitive, flowable and desirable than del list. Or maybe I haven't had enough coffee this morning. I'd go as far as saying all container objects should have a clear method. This is the reason why those Rubyist whine about Python being inconsistent and not being OO enough. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
[Mystilleef] Lists should have a clear method. But what's shocking is that it doesn't seem obvious to others. list.clear() is a whole lot more readable, intuitive, flowable and desirable than [the alternatives] +1 to all of that. -- Richie Hindle [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
A perspective that I haven't seen raised here is inheritance. I often say mylist = [] if I'm done with the current contents and just want a fresh list. But the cases where I have really needed list.clear [and laboriously looked for it and ended up with del l[:] were when the object was my own class that inherits from list, adding some state and other functionality. Of course I *could* have added my own 'clear' function member, but I *wanted* it to behave like a standard python list in it's role of maintaining a sequence of processing steps. So, I end up doing del current_process[:] which, IMO, looks a bit queer, especially when current_process object is a fairly elaborate data object. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger wrote: [Steven Bethard] I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? Something this small doesn't need a PEP. I'll just send a note to Guido asking for a pronouncement. Thanks. It'd be nice to have something to refer to the next time this comes up. Here's a draft list of pros and cons (any changes or suggestions are welcome): Pros: - * s.clear() is more obvious in intent * easier to figure-out, look-up, and remember than either s[:]=[] or del s[:] * parallels the api for dicts, sets, and deques (increasing the expecation that lists will too) * the existing alternatives are a bit perlish I'd snip the one below. It doesn't really contribute anything, and was sarcastic in the first place. ;) * the OP is shocked, SHOCKED that python got by for 16 years without list.clear() Cons: - * makes the api fatter (there are already two ways to do it) * expanding the api makes it more difficult to write classes that can be polymorphically substituted for lists * learning slices is basic to the language (this lesson shouldn't be skipped) * while there are valid use cases for re-using lists, the technique is already overused for unsuccessful attempts to micro-optimize (creating new lists is surprisingly fast) In fairness, if we're going to drop the SHOCKED comment, we should drop the first two clauses of the point below (but the relevant idiom part is clearly a good point). * the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) Other than those minor edits, I think you've pretty much gotten everything. I look forward to a pronouncement. Thanks again! STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Mel Wilson wrote: for item in seq: L.append(item) Both extend and append have one-line slice equivalents, except that the equivalents have to keep referring to the length of the list.. (have to keep finding the len function.) fwiw, the *tutorial* defines append and extend in terms of slicing... /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
* the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) Is this sort of editorialising fair, or just a way of not-so-subtly encouraging Guido to reject the whole idea, now and forever? Bah. Guido is not stupid, nor easily misled. Both the pros and cons were quipped with abrupt perjoratives so the bullet points could be stated succinctly and with a bit of levity. The translation to verbose, soft, politically correct statements is self-evident. request is inane -- A generation of python programmers has found list clearing to be like other parts of the language that you get used to very quickly and do not prove to be a problem in practice. The request is in the same category as others which challenge api choices made 16 years ago; in particular, the decision to have compact APIs where the named methods do not duplicate functionality provided by syntax using operators and keywords. The request is less of a bug report and more a rejection of Guido's sense of design and his subsequent experience using his own language. underlying problem is trivial -- Books such as the Python Pocket Reference or Python in a Nutshell are able to cover this idiom with just a single sentence. Once known and understood, the application of the current-way-to-do-it is immediate, compact, and effective. the existing alternatives are a bit perlish -- Both alternatives involve a bit of inventiveness in combining two ideas (either the del keyword and its interaction with slicing notation or the assignment of an emtpy list to a slice). Each approach has a visual appearance of being a syntax trick. The effect contrasts with much of the rest of the language where it is usually possible to write code is a way that can be read and understood by non-python programmers. The existing solution trades readability for the succinctness of a compact syntactical idiom. A list.clear method will make deleting items from a list more OO, consistent with almost everything else you do to lists, and less procedural. This is especially true if clear() takes an optional index (or two), allowing sections of the list to be cleared, not just the entire list. Don't shoot yourself in the foot here. If you want to advocate list.clear(), then you're hurting your chances by pushing for it to take an optional argument. Essentially, this amounts to an unwillingness to use the del-keyword and to duplicate its functionality with a named method. Also, in the python-dev world, making something more OO is neither a virtue nor a vice. It is better to argue for rich functionality, succeptibility to errors, or dramatic improvements of existing real-world code. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger wrote: Also, in the python-dev world, making something more OO is neither a virtue nor a vice. except that arguments along the line of if the syntax is not obj.method(), it's not OO enough are likely to be mostly ignored. (nobody's going to be impressed by yet another len(obj) isn't OO variant) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Fredrik Lundh [EMAIL PROTECTED] wrote: except that arguments along the line of if the syntax is not obj.method(), it's not OO enough are likely to be mostly ignored. (nobody's going to be impressed by yet another len(obj) isn't OO variant) Does that suggest that what's needed is clear(obj) and __clear__ methods? 8-) -- \S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/ ___ | Frankly I have no feelings towards penguins one way or the other \X/ |-- Arthur C. Clarke her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
[Dan Christensen] It's true that this runs at the same speed as the del variants on my machine. That's not too surprising to me, but I still don't understand why the del variants are more than 5% faster than the first version. Understanding it involves looking at implementation specific details such as the overallocation scheme for growing lists and the performance of your machine's underlying memory allocator. Once this is understood, is it something that could be optimized? It's pretty common to rebind a variable to a new value, and if this could be improved 5%, that would be cool. Sorry, that's not how the language works. Something like a=range(10) means: * build a NEW list for 10 elements * THEN assign it to the variable a * which THEN reduces the ref count to the previous binding for a * and THEN IF the ref count is zero, free the list previously bound to a In other words, if a is already bound to a large list, then the above assignment necessarily creates a second, non-overlapping list in memory. However, if you write a=None;a=range(10), then the original list gets freed BEFORE the new list is created and the system has a chance to re-use that large, contiguous block of memory. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen [EMAIL PROTECTED] writes: [...] Then it's a good reason we had this thread, so you could learn something *crucial* to understanding Python and writing non-buggy code: name binding versus variables which occupy fixed memory locations like in some other languages. This has to be by far the most frequent area that newcomer's trip up. But that's another story... I, as a newcomer, don't have much trouble understanding the binding vs the assignment by themselves. What does somewhat confuse is dual role of the = operator, -- sometimes it means bind and other times it means assign, right? For me it seems that the language would be less error-prone and easier to grok if these two operations had different names/syntax (thinking about lisp let vs set), though it seems to be too drastic change even for Python3000. -- Sergei. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Em Sex, 2006-04-14 às 09:17 +0400, Sergei Organov escreveu: I, as a newcomer, don't have much trouble understanding the binding vs the assignment by themselves. What does somewhat confuse is dual role of the = operator, -- sometimes it means bind and other times it means assign, right? For me it seems that the language would be less error-prone and easier to grok if these two operations had different names/syntax (thinking about lisp let vs set), though it seems to be too drastic change even for Python3000. The = operator *always* binds. -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; the other benchmarks you need only a single block of memory (*). I don't follow you here :) Concluding from here gets difficult - you would have to study the malloc implementation to find out whether it works better in one case over the other. That's not the case here. The following program prints the same addresses whether you comment or uncomment del - ids = range(10) for i in xrange(10): x = range(10) #del x[:] ids[i] = id(x) print ids Could also be an issue of processor cache: one may fit into the cache, but the other may not. That's not the reason, since the same time difference happens with smaller arrays. I think the difference is how items are deallocated in these two cases. Calling del invokes list_ass_slice that deallocates from 0 to end whereas ordinary removal of a list was changed to backward iteration (the reason is in the comment: /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces thrashing when a *very* large list is created and immediately deleted. */ Usually iterating from low addresses to higher addresses is better for CPU. On my CPU (Pentium M, 1.7Ghz) it's 20% faster: Here is my results: C:\pypython -mtimeit x = range(1); del x[:] 1000 loops, best of 3: 213 usec per loop C:\pypython -mtimeit x = range(1); del x 1000 loops, best of 3: 257 usec per loop C:\pypython -mtimeit x = range(1); 1000 loops, best of 3: 258 usec per loop C:\pypython -mtimeit x = range(1000); del x[:] 1 loops, best of 3: 21.4 usec per loop C:\pypython -mtimeit x = range(1000); del x 1 loops, best of 3: 25.2 usec per loop C:\pypython -mtimeit x = range(1000); 1 loops, best of 3: 25.6 usec per loop I don't have a development environment on my computer so I can't test my thoughts. I could be wrong about the reason. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; the other benchmarks you need only a single block of memory (*). Concluding from here gets difficult - you would have to study the malloc implementation to find out whether it works better in one case over the other. Could also be an issue of processor cache: one may fit into the cache, but the other may not. Addition to the previous message. Now I follow you :) There are indeed two arrays and cache seems to be the second reason for slowdown, but iterating backwards is also contributing to slowdown. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Felipe Almeida Lessa wrote: Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu: On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; Er, what new list? I see only one list, x = range(10), which is merely created then nothing done to it. Have I missed something? He's talking about the garbage collector. To be exact the reason for two array is timeit.py. It doesn't place the code to time into a separate namespace but injects it into a for loop, so the actual code timed is: for _i in _it: x = range(10) and that makes two arrays with 100.000 items exist for a short time starting from second iteration. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Wed, 12 Apr 2006 00:33:29 -0700, Serge Orlov wrote: Felipe Almeida Lessa wrote: Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu: On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; Er, what new list? I see only one list, x = range(10), which is merely created then nothing done to it. Have I missed something? He's talking about the garbage collector. To be exact the reason for two array is timeit.py. It doesn't place the code to time into a separate namespace but injects it into a for loop, so the actual code timed is: for _i in _it: x = range(10) and that makes two arrays with 100.000 items exist for a short time starting from second iteration. But that is precisely the same for the other timeit tests too. for _i in _it: x = range(10) del x[:] etc. The question remains -- why does it take longer to do X than it takes to do X and then Y? -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote: But that is precisely the same for the other timeit tests too. for _i in _it: x = range(10) Allocate list. Allocate ob_item array to hold pointers to 1 objects Allocate 99900 integer objects setup list del x[:] Calls list_clear which: decrements references to 99900 integer objects, freeing them frees the ob_item array ... next time round ... x = range(10) Allocate new list list. Allocate ob_item array probably picks up same memory block as last time Allocate 99900 integer objects, probably reusing same memory as last time setup list etc. The question remains -- why does it take longer to do X than it takes to do X and then Y? for _i in _it: ... x = range(10); Allocate list. Allocate ob_item array to hold pointers to 1 objects Allocate 99900 integer objects setup list ... next time round ... Allocate another list allocate a second ob_item array allocate another 99900 integer objects setup the list then deletes the original list, decrements and releases original integers, frees original ob_item array. Uses twice as much everything except the actual list object. The actual work done is the same but I guess there are likely to be more cache misses. Also there is the question whether the memory allocation does or does not manage to reuse the recently freed blocks. With one large block I expect it might well end up reusing it, with two large blocks being freed alternately it might not manage to reuse either (but that is just a guess and maybe system dependant). -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote: But name.clear() meaning mutate the object referenced by name to the empty state is a very natural candidate for a method, and I don't understand why lists shouldn't have it. Funny this even comes up, because I was just trying to 'clear' a list the other day. But it sounds like it's been an issue for a while. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote: On Tue, 11 Apr 2006 14:49:04 -0700, Ville Vainio wrote: John Salerno wrote: Thanks guys, your explanations are really helpful. I think what had me confused at first was my understanding of what L[:] does on either side of the assignment operator. On the left, it just chooses those elements and edits them in place; on the right, it makes a copy of that list, right? (Which I guess is still more or less *doing* the same thing, just for different purposes) Interestingly, if it was just a clear method nobody would be confused. Even more importantly, you could say help(list.clear) and learn something useful, instead of trying help(del) and getting a syntax error. [snip more reasons to add list.clear()] I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven Bethard wrote: I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? What are the usual arguments against adding it? -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Steven Bethard wrote: I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? What are the usual arguments against adding it? That there should be one obvious way to do it. Yes, I know that it can be debated whether del x[:] is obvious, and fortunately I'm not the one to decide wink. Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Georg Brandl wrote: John Salerno wrote: Steven Bethard wrote: I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? What are the usual arguments against adding it? That there should be one obvious way to do it. Yes, I know that it can be debated whether del x[:] is obvious, and fortunately I'm not the one to decide wink. It's so obvious, I couldn't even find it in the tutorial the other day when this thread came up. It's a wonder I ever learned that one myself, and I don't know where I saw it. .clear(), on the other hand, would clearly be easy to add/find in the tutorial... and we'd all save time answering the question over and over around here (valuable time we would then be able to using telling people who hadn't, to go and read the tutorial! :-) ). -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Steven Bethard wrote: I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? What are the usual arguments against adding it? I think one is that it's very rare to need it. Most of the time, just rebinding the name to a new empty list is easier, and possibly faster. The main problem with that approach is that in multithreaded code that can be a source of subtle bugs. On the other hand, most things can be a source of subtle bugs in multithreaded code, so maybe that's not a good reason to add clear(). Saving us time answering the question repeatedly (and trying to justify the current state of affairs) might be justification enough... -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Steven Bethard wrote: I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? What are the usual arguments against adding it? I think most of the other folks have already given you the answers, but the main ones I remember: (1) There are already at least two ways of spelling it. We don't need another. (2) Clearing a list is a pretty uncommon operation. Usually just creating a new list is fine (and easier). Disclaimer: I'm happy with adding list.clear(), so it's quite possible I'm misrepresenting the argument here. I guess that's just all the more reason to have a PEP for it: to lay out all the pros and cons. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; Er, what new list? I see only one list, x = range(10), which is merely created then nothing done to it. Have I missed something? See Duncan's explanation. This code is run many times, allocating many lists. However, only two different lists exist at any point at time. A Python list consists of two memory blocks: the list proper (a few bytes), plus the guts, i.e. a variable-sized block of pointers to the objects. del x[:] frees the guts. I understood Felipe to be asking, why does it take longer to just create a list, than it takes to create a list AND then do something to it? Actually, the same code (deallocation of all integers and the list blocks) appears in either case. However, in the one case it is triggered explicitly, and before the new allocation; in the other case, the allocation of the new objects occurs before the old ones are released. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Ville Vainio wrote: Fredrik Lundh wrote: because Python already has a perfectly valid way to clear a list, perhaps ? del l[:] Ok. That's pretty non-obvious but now that I've seen it I'll probably remember it. I did a stupid while l: l.pop() loop myself. Actually, it's in the Library Reference (that we keep under our pillows) section 2.3.6.4 Mutable Sequence Types. Cheers,Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
[Steven Bethard] I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? Something this small doesn't need a PEP. I'll just send a note to Guido asking for a pronouncement. Here's a draft list of pros and cons (any changes or suggestions are welcome): Pros: - * s.clear() is more obvious in intent * easier to figure-out, look-up, and remember than either s[:]=[] or del s[:] * parallels the api for dicts, sets, and deques (increasing the expecation that lists will too) * the existing alternatives are a bit perlish * the OP is shocked, SHOCKED that python got by for 16 years without list.clear() Cons: - * makes the api fatter (there are already two ways to do it) * expanding the api makes it more difficult to write classes that can be polymorphically substituted for lists * learning slices is basic to the language (this lesson shouldn't be skipped) * while there are valid use cases for re-using lists, the technique is already overused for unsuccessful attempts to micro-optimize (creating new lists is surprisingly fast) * the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Em Qua, 2006-04-12 às 12:40 -0700, Raymond Hettinger escreveu: * the existing alternatives are a bit perlish I love this argument =D! perlish... lol... Cheers, -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
[Felipe Almeida Lessa] I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? [Dan Christensen] I get similar behaviour. No idea why. It is an effect of the memory allocator and fragmentation. The first builds up a list with increasingly larger sizes. It periodically cannot grow in-place because something is in the way (some other object) so it needs to move its current entries to another, larger block and grow from there. In contrast, the other entries are reusing a the previously cleared out large block. Just for grins, replace the first with 'x=None; x=range(10)' The assignment to None frees the reference to the previous list and allows it to be cleared so that its space is immediately available to the new list being formed by range(). -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger wrote: Cons: - * learning slices is basic to the language (this lesson shouldn't be skipped) And yet it doesn't appear to be in the tutorial. I could have missed it, but I've looked in a number of the obvious places, without actually going through it (again) from start to finish. Also, googling for slice site:docs.python.org, you have to go to the *sixth* entry before you can find the first mention of del x[:] and what it does. I think given the current docs it's possible to learn all kinds of things about slicing and still not make the non-intuitive leap that del x[slice] is actually how you spell delete contents of list in-place. * while there are valid use cases for re-using lists, the technique is already overused for unsuccessful attempts to micro-optimize (creating new lists is surprisingly fast) Not just valid use-cases, but ones for which x = [] is entirely buggy, yet not obviously so, especially to newcomers. * the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) The first phrase is insulting and untrue, the second merely untrue (as even your own list of pros and cons shows), and the last completely valid and highly relevant, yet not overriding. -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger wrote: * easier to figure-out, look-up, and remember than either s[:]=[] or del s[:] Easier is an understatement - it's something you figure out automatically. When I want to do something w/ an object, looking at its methods (via code completion) is the very first thing. * the OP is shocked, SHOCKED that python got by for 16 years without list.clear() I'm sure you realize I was being sarcastic... * learning slices is basic to the language (this lesson shouldn't be skipped) Assigning to slices is much less important, and is something I always never do (and hence forget). * the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) I understand that these are the main arguments. However, as it stands there is no one *obvious* way to clear a list in-place. I agree that it's rare to even need it, but when you do a it's a little bit of a surprise. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Ville Vainio wrote: Assigning to slices is much less important, and is something I always never do (and hence forget). ALMOST never, of course. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: [Steven Bethard] I think these are all good reasons for adding a clear method, but being that it has been so hotly contended in the past, I don't think it will get added without a PEP. Anyone out there willing to take out the best examples from this thread and turn it into a PEP? Something this small doesn't need a PEP. I'll just send a note to Guido asking for a pronouncement. Here's a draft list of pros and cons (any changes or suggestions are welcome): Pros: - * s.clear() is more obvious in intent Serious question: Should it work more like s=[] or more like s[:]=[]. I'm assuming the latter, but the fact that there is a difference is an argument for not hiding this operation behind some syntactic sugar. Alan -- Defendit numerus -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Wed, 12 Apr 2006 12:40:52 -0700, Raymond Hettinger wrote: Something this small doesn't need a PEP. I'll just send a note to Guido asking for a pronouncement. Raymond, if you're genuinely trying to help get this sorted in the fairest, simplest way possible, I hope I speak for everyone when I say thank you, your efforts are appreciated. But there is this: * the request is inane, the underlying problem is trivial, and the relevant idiom is fundamental (api expansions should be saved for rich new functionality and not become cluttered with infrequently used redundant entries) Is this sort of editorialising fair, or just a way of not-so-subtly encouraging Guido to reject the whole idea, now and forever? Convenience and obviousness are important for APIs -- that's why lists have pop, extend and remove methods. The only difference I can see between a hypothetical clear and these is that clear can be replaced with a one-liner, while the others need at least two, e.g. for extend: for item in seq: L.append(item) Here is another Pro for your list: A list.clear method will make deleting items from a list more OO, consistent with almost everything else you do to lists, and less procedural. This is especially true if clear() takes an optional index (or two), allowing sections of the list to be cleared, not just the entire list. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote: Serious question: Should it work more like s=[] or more like s[:]=[]. I'm assuming the latter, but the fact that there is a difference is an argument for not hiding this operation behind some syntactic sugar. Er, I don't see how it can possibly work like s = []. That merely reassigns a new empty list to the name s, it doesn't touch the existing list (which may or may not be garbage collected soon/immediately afterwards). As far as I know, it is impossible -- or at least very hard -- for an object to know which namesspaces it is in, so it can reassign one name but not the others. Even if such a thing was possible, I think it is an absolutely bad idea. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
In article [EMAIL PROTECTED], Steven D'Aprano [EMAIL PROTECTED] wrote: On Wed, 12 Apr 2006 15:36:47 -0700, Alan Morgan wrote: Serious question: Should it work more like s=[] or more like s[:]=[]. I'm assuming the latter, but the fact that there is a difference is an argument for not hiding this operation behind some syntactic sugar. Er, I don't see how it can possibly work like s = []. That merely reassigns a new empty list to the name s, it doesn't touch the existing list (which may or may not be garbage collected soon/immediately afterwards). Right. I was wondering what would happen in this case: s=[1,2,3] t=s s.clear() t # [] or [1,2,3]?? If you know your basic python it is obvious what would happen if you do s=[] or s[:]=[] instead of s.clear() and I guess it is equally obvious which one s.clear() must mimic. I'm still not used to dealing with mutable lists. Alan -- Defendit numerus -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Alan Morgan wrote: Right. I was wondering what would happen in this case: s=[1,2,3] t=s s.clear() t # [] or [1,2,3]?? If you know your basic python it is obvious what would happen if you do s=[] or s[:]=[] instead of s.clear() and I guess it is equally obvious which one s.clear() must mimic. I'm still not used to dealing with mutable lists. If you know your basic python :-), you know that s[:] = [] is doing the only thing that s.clear() could possibly do, which is changing the contents of the list which has the name s bound to it (and which might have other names bound to it, just like any object in Python). It *cannot* be doing the same as s=[] which does not operate on the list but creates an entirely new one and rebinds the name s to it. The only possible answer for your question above is t is s and t == [] because you haven't rebound the names. -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven D'Aprano wrote:
Convenience and obviousness are important for APIs -- that's why lists
have pop, extend and remove methods. The only difference I can see between
a hypothetical clear and these is that clear can be replaced with a
one-liner, while the others need at least two, e.g. for extend:
for item in seq:
L.append(item)
It's not even clear that extend needs two lines:
s = range(5)
more = list('abc')
s[:] = s + more
s
[0, 1, 2, 3, 4, 'a', 'b', 'c']
Okay, it's not obvious, but I don't think s[:]=[] is really any more
obvious as a way to clear the list.
Clearly .extend() needs to be removed from the language as it is an
unnecessary extension to the API using slicing, which everyone should
already know about...
-Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
In article [EMAIL PROTECTED], Peter Hansen [EMAIL PROTECTED] wrote: Alan Morgan wrote: Right. I was wondering what would happen in this case: s=[1,2,3] t=s s.clear() t # [] or [1,2,3]?? If you know your basic python it is obvious what would happen if you do s=[] or s[:]=[] instead of s.clear() and I guess it is equally obvious which one s.clear() must mimic. I'm still not used to dealing with mutable lists. If you know your basic python :-), you know that s[:] = [] is doing the only thing that s.clear() could possibly do, Ah, but if you know your basic python then you wouldn't be looking for s.clear() in the first place; you'd just use s[:]=[] (or s=[], whichever is appropriate). IOW, the people most in need of s.clear() are those least likely to be able to work out what it is actually doing. Personally, it seems more *reasonable* to me, a novice python user, for s.clear() to behave like s=[] (or, perhaps, for s=[] and s[:]=[] to mean the same thing). The fact that it can't might be an argument for not having it in the first place. Alan -- Defendit numerus -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Peter Hansen [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
It's not even clear that extend needs two lines:
s = range(5)
more = list('abc')
s[:] = s + more
s
[0, 1, 2, 3, 4, 'a', 'b', 'c']
This is not the same as list.extend because it makes a separate
intermediate list instead of doing the extension completely in place.
However, the following does mimic .extend.
s=range(5)
s[len(s):] = list('abc')
s
[0, 1, 2, 3, 4, 'a', 'b', 'c']
So. at the cost of writing and calling len(s), you are correct that .extend
is not necessary.
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Raymond Hettinger [EMAIL PROTECTED] writes: Felipe Almeida Lessa writes: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? It is an effect of the memory allocator and fragmentation. The first builds up a list with increasingly larger sizes. I don't see what you mean by this. There are many lists all of the same size. Do you mean some list internal to the memory allocator? It periodically cannot grow in-place because something is in the way (some other object) so it needs to move its current entries to another, larger block and grow from there. In contrast, the other entries are reusing a the previously cleared out large block. Just for grins, replace the first with 'x=None; x=range(10)' The assignment to None frees the reference to the previous list and allows it to be cleared so that its space is immediately available to the new list being formed by range(). It's true that this runs at the same speed as the del variants on my machine. That's not too surprising to me, but I still don't understand why the del variants are more than 5% faster than the first version. Once this is understood, is it something that could be optimized? It's pretty common to rebind a variable to a new value, and if this could be improved 5%, that would be cool. But maybe it wouldn't affect anything other than such micro benchmarks. Dan -- http://mail.python.org/mailman/listinfo/python-list
list.clear() missing?!?
I tried to clear a list today (which I do rather rarely, considering that just doing l = [] works most of the time) and was shocked, SHOCKED to notice that there is no clear() method. Dicts have it, sets have it, why do lists have to be second class citizens? -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Ville Vainio wrote: I tried to clear a list today (which I do rather rarely, considering that just doing l = [] works most of the time) and was shocked, SHOCKED to notice that there is no clear() method. Dicts have it, sets have it, why do lists have to be second class citizens? because Python already has a perfectly valid way to clear a list, perhaps ? del l[:] (lists are not mappings, so the duck typing argument don't really apply here.) /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Fredrik Lundh wrote: I tried to clear a list today (which I do rather rarely, considering that just doing l = [] works most of the time) and was shocked, SHOCKED to notice that there is no clear() method. Dicts have it, sets have it, why do lists have to be second class citizens? because Python already has a perfectly valid way to clear a list, perhaps ? del l[:] Ok. That's pretty non-obvious but now that I've seen it I'll probably remember it. I did a stupid while l: l.pop() loop myself. (lists are not mappings, so the duck typing argument don't really apply here.) I was thinking of list as a mutable collection, and clear() is certainly a very natural operation for them. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Ville Vainio wrote: I tried to clear a list today (which I do rather rarely, considering that just doing l = [] works most of the time) and was shocked, SHOCKED to notice that there is no clear() method. Dicts have it, sets have it, why do lists have to be second class citizens? This gets brought up all the time (search the archives for your favorite), but your options are basically (renaming your list to lst for readability) one of:: del lst[:] lst[:] = [] or if you don't need to modify the list in place, lst = [] Personally, I tend to go Fredrik's route and use the first. If you feel really strongly about this though, you might consider writing up a PEP. It's been contentious enough that there's not much chance of getting a change without one. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Em Ter, 2006-04-11 às 10:42 -0600, Steven Bethard escreveu: one of:: del lst[:] lst[:] = [] or if you don't need to modify the list in place, lst = [] Personally, I tend to go Fredrik's route and use the first. I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? Cheers, -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven Bethard wrote: If you feel really strongly about this though, you might consider writing up a PEP. It's been contentious enough that there's not much chance of getting a change without one. No strong feelings here, and I'm sure greater minds than me have already hashed this over sufficiently. It's just that, when I have an object, and am wondering how I can clear it, I tend to look what methods it has first and go to google looking for idioms second. Perhaps clear method could be added that raises PedagogicException(Use del lst[:], stupid!)? *ducks* -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; the other benchmarks you need only a single block of memory (*). Concluding from here gets difficult - you would have to study the malloc implementation to find out whether it works better in one case over the other. Could also be an issue of processor cache: one may fit into the cache, but the other may not. Regards, Martin (*) plus, you also need the integer objects twice. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Ville Vainio wrote: It's just that, when I have an object, and am wondering how I can clear it, I tend to look what methods it has first and go to google looking for idioms second. I guess del on a list is not that common, so people tend to not know that it works on lists (and slices!), too. It's too bad that lists have a pop() method these days, so people can do x.pop() even if they don't need the value, instead of doing del x[-1]. I don't think I ever needed to del a slice except for clearing the entire list (and I don't need to do that often, either - I just throw the list away). Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Steven Bethard wrote: lst[:] = [] lst = [] What's the difference here? -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Em Ter, 2006-04-11 às 17:56 +, John Salerno escreveu: Steven Bethard wrote: lst[:] = [] lst = [] What's the difference here? lst[:] = [] makes the specified slice become []. As we specified :, it transforms the entire list into []. lst = [] assigns the value [] to the variable lst, deleting any previous one. This might help: lst = range(10) id(lst), lst (-1210826356, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) lst[:] = [] id(lst), lst (-1210826356, []) lst = range(10) id(lst), lst (-1210844052, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) lst = [] id(lst), lst (-1210826420, []) You see? lst[:] removes all elements from the list that lst refers to, while lst = [] just creates a new list and discard the only one. The difference is, for example: lst = range(3) x = [lst, lst, lst] x [[0, 1, 2], [0, 1, 2], [0, 1, 2]] lst[:] = [] x [[], [], []] lst = range(3) x = [lst, lst, lst] x [[0, 1, 2], [0, 1, 2], [0, 1, 2]] lst = [] x [[0, 1, 2], [0, 1, 2], [0, 1, 2]] HTH, -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Steven Bethard wrote: lst[:] = [] lst = [] What's the difference here? L[:]= modifies the object in place, L=[] binds the variable to a new object. compare and contrast: L = [a, b, c] M = L L ['a', 'b', 'c'] M ['a', 'b', 'c'] L is M True L[:] = [] L [] M [] L is M True L = [a, b, c] M = L L ['a', 'b', 'c'] M ['a', 'b', 'c'] L = [] L [] M ['a', 'b', 'c'] L is M False /F -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Steven Bethard wrote: lst[:] = [] lst = [] What's the difference here? lst = [1,2,3] lst2 = lst lst[:] = [] lst2 [] lst = [1,2,3] lst2 = lst lst = [] lst2 [1, 2, 3] Duncan -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Felipe Almeida Lessa wrote: You see? lst[:] removes all elements from the list that lst refers to, while lst = [] just creates a new list and discard the only one. The difference is, for example: Thanks, your explanation was great! -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Fredrik Lundh wrote: John Salerno wrote: Steven Bethard wrote: lst[:] = [] lst = [] What's the difference here? L[:]= modifies the object in place, L=[] binds the variable to a new object. compare and contrast: Thanks guys, your explanations are really helpful. I think what had me confused at first was my understanding of what L[:] does on either side of the assignment operator. On the left, it just chooses those elements and edits them in place; on the right, it makes a copy of that list, right? (Which I guess is still more or less *doing* the same thing, just for different purposes) -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
John Salerno wrote: Thanks guys, your explanations are really helpful. I think what had me confused at first was my understanding of what L[:] does on either side of the assignment operator. On the left, it just chooses those elements and edits them in place; on the right, it makes a copy of that list, right? (Which I guess is still more or less *doing* the same thing, just for different purposes) Interestingly, if it was just a clear method nobody would be confused. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Tue, 11 Apr 2006 14:49:04 -0700, Ville Vainio wrote: John Salerno wrote: Thanks guys, your explanations are really helpful. I think what had me confused at first was my understanding of what L[:] does on either side of the assignment operator. On the left, it just chooses those elements and edits them in place; on the right, it makes a copy of that list, right? (Which I guess is still more or less *doing* the same thing, just for different purposes) Interestingly, if it was just a clear method nobody would be confused. Even more importantly, you could say help(list.clear) and learn something useful, instead of trying help(del) and getting a syntax error. help(del) File stdin, line 1 help(del) ^ SyntaxError: invalid syntax I know Python isn't a purely OOP language, but in my opinion using statements like del should be avoided when there is an easy and natural OO way of doing it. Something like name.del() which means delete the reference to name in name's namespace feels wrong, and may not even be possible with Python's object model, so del name is a natural way to do it. But name.clear() meaning mutate the object referenced by name to the empty state is a very natural candidate for a method, and I don't understand why lists shouldn't have it. For lists, it would be natural to have a hypothetical clear() method accept an index or slice as an argument, so that these are equivalent: del L[:] = L.clear() del L[n] = L.clear(n) del L[a:b] = L.clear(slice(a, b)) # untested reference implementation: class ClearableList(list): def clear(self, obj=None): if obj is None: obj = slice(0, len(self)) if isinstance(obj, int): self.__delitem__(obj) elif isinstance(obj, slice): self.__delslice__(obj.start, obj.stop) else: raise TypeError -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; Er, what new list? I see only one list, x = range(10), which is merely created then nothing done to it. Have I missed something? I understood Felipe to be asking, why does it take longer to just create a list, than it takes to create a list AND then do something to it? -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu: On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote: Felipe Almeida Lessa wrote: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? In the first benchmark, you need space for two lists: the old one and the new one; Er, what new list? I see only one list, x = range(10), which is merely created then nothing done to it. Have I missed something? He's talking about the garbage collector. Talking about the GC, do you want to see something *really* odd? $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x = range(10); ' 100 loops, best of 3: 13 msec per loop $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x = range(10); del x[:]' 100 loops, best of 3: 8.19 msec per loop $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x = range(10); x[:] = []' 100 loops, best of 3: 8.16 msec per loop $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x = range(10); del x' 100 loops, best of 3: 8.3 msec per loop But in this case I got the answer (I think): - When you explicitly delete the objects, the GC already know that it can be collected, so it just throw the objects away. - When we let the x variable continue to survive, the GC has to look at all the 11 objects to see if they can be collected -- just to see that it can't. Also, IIRC del x is slower than x = [] because removing a name from the namespace is more expensive than just assigning something else to it. Right? I understood Felipe to be asking, why does it take longer to just create a list, than it takes to create a list AND then do something to it? I see dead people... ;-) -- Felipe. -- http://mail.python.org/mailman/listinfo/python-list
Re: list.clear() missing?!?
Felipe Almeida Lessa [EMAIL PROTECTED] writes: I love benchmarks, so as I was testing the options, I saw something very strange: $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.7 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.35 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.36 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.46 msec per loop Why the first benchmark is the slowest? I don't get it... could someone test this, too? I get similar behaviour. No idea why. $ python2.4 -mtimeit 'x = range(10); ' 100 loops, best of 3: 6.99 msec per loop $ python2.4 -mtimeit 'x = range(10); del x[:]' 100 loops, best of 3: 6.49 msec per loop $ python2.4 -mtimeit 'x = range(10); x[:] = []' 100 loops, best of 3: 6.47 msec per loop $ python2.4 -mtimeit 'x = range(10); del x' 100 loops, best of 3: 6.6 msec per loop Dan -- http://mail.python.org/mailman/listinfo/python-list
