Re: Multi-dimensional list initialization
On Nov 9, 11:37 am, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: On Fri, 09 Nov 2012 17:07:09 +1100, Chris Angelico wrote: On Fri, Nov 9, 2012 at 12:39 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x Can you enlighten us as to how this is better than either: x := -x or x := 0 - x ? I'm not seeing it. I'm hoping that Mark intended it as an example of crappy code he has spotted in some other language rather than a counter-example of something you would do. To be pedantic... there may very well be some (rare) cases where you actually do want x -= x rather than just x = 0. Consider the case where x could be an INF or NAN. Then x -= x should give x = NAN rather than zero. That may be desirable in some cases. In x86 assembler mov ax, 0 is 4 bytes sub ax, ax is 2 and therefore better (at least for those brought up on Peter Norton); the most common being xor ax, ax -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Sat, Nov 10, 2012 at 2:05 AM, rusi rustompm...@gmail.com wrote: In x86 assembler mov ax, 0 is 4 bytes Three bytes actually, B8 00 00 if my memory hasn't failed me. BA for DX, B9 ought to be BX and BB CX, I think. But yes, the xor or sub is two bytes and one clock. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
RE: Multi-dimensional list initialization
Dennis Lee Bieber wrote: On Fri, 9 Nov 2012 17:07:09 +1100, Chris Angelico ros...@gmail.com declaimed the following in gmane.comp.python.general: On Fri, Nov 9, 2012 at 12:39 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x Can you enlighten us as to how this is better than either: x := -x or x := 0 - x Of course, if one has a language that, for some reason, evaluates right-to-left (APL, anyone), then x := x - x - x becomes x := x - 0 Is that not the same as x:=-x? ~Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Prasad, Ramit wrote: Dennis Lee Bieber wrote: Of course, if one has a language that, for some reason, evaluates right-to-left (APL, anyone), then x := x - x - x becomes x := x - 0 Is that not the same as x:=-x? No, its the same as 'x = x'. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/07/2012 11:09 PM, Ian Kelly wrote: On Wed, Nov 7, 2012 at 8:13 PM, Andrew Robinson andr...@r3dsolutions.com wrote: OK, and is this a main use case? (I'm not saying it isn't I'm asking.) I have no idea what is a main use case. Well, then we can't evaluate if it's worth keeping a list multiplier around at all. You don't even know how it is routinely used. FYI, the Python devs are not very fond of adding new keywords. Any time a new keyword is added, existing code that uses that word as a name is broken. 'ini' is particularly bad, because 1) it's not a word, and 2) it's the name of a common type of configuration file and is probably frequently used as a variable name in relation to such files. Fine; Its a keyword TBD then; I should have said 'foo'. in is worse than ini, ini is worse than something else -- at the end of the rainbow, maybe there is something values = zip( samples, [ lambda:times, ini xrange(num_groups) ] ) if len(values) len(times) * num_groups How is this any better than the ordinary list comprehension I already suggested as a replacement? For that matter, how is this any better than list multiplication? You _asked it to implement_ a list multiplication of the traditional kind; By doing copies by *REFERENCE*; so of course it's not better. My intentions were for copying issues, not-non copying ones. Your basic complaint about list multiplication as I understand it is that the non-copying semantics are unintuitive. No. 1) My basic complaint is that people (I think from watching) primarily use it to make initializer lists, and independent mutable place holders; List multiplication doesn't do that well. 2) If it is indeed very rare (as D'Aprano commented) then -- it has a second defect in looking to casual inspection to be the same as vector multiplication; which opacifies which operation is being done when matrix packages are potentially being used. Well, the above is even less intuitive. It is excessively complicated and almost completely opaque. If I were to come across it outside the context of this thread, I would have no idea what it is meant to be doing. Nor would I *know* what this list multiplier look alike does [1,2,3]*aValue without checking to see if someone imported a vector library and the variable aValue has a special multiplication operator. As an aside, how would you do the lambda inside a list comprehension? As a general rule, I wouldn't. I would use map instead. OK: Then copy by reference using map: values = zip( map( lambda:times, xrange(num_groups) ) ) if len(values) len(times) * num_groups ... Done. It's clearer than a list comprehension and you still really don't need a list multiply. I''m not going to bother explaining what the construction I offered would be really good at. It's pointless to explain to the disinterested. Thak constructs a list of 10 functions and never calls them. If you want to actually call the lambda, then: Yep, I was very tired. slice.indices() has nothing to do with it. Indexing a sequence and calling the .indices() method on a slice are entirely different operations. Yes, but you're very blind to history and code examples implementing the slice operation. slice usually depends on index; index does not depend on slice. Slice is suggested to be implemented by multiple calls to single indexes in traditional usage and documentation. The xrange(,,)[:] implementation breaks the tradition, because it doesn't call index multiple times; nor does it return a result equivalent identical to doing that. It's different. period. You're not convincing in the slightest by splitting hairs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Nov 7, 2012, at 11:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: On 11/07/2012 04:00 PM, Steven D'Aprano wrote: Andrew, it appears that your posts are being eaten or rejected by my ISP's news server, because they aren't showing up for me. Possibly a side- effect of your dates being in the distant past? Date has been corrected since two days ago. It will remain until a reboot Ignorance, though, might be bliss... Every now and again I come across somebody who tries to distinguish between call by foo and pass by foo, but nobody has been able to explain the difference (if any) to me. I think the Call by foo came into vogue around the time of C++; Eg: It's in books like C++ for C programmers; I never saw it used before then so I *really* don't know for sure… Just as an aside - there is a famous quote from Niklaus Wirt, who, when asked how he pronounced his name, is said to have replied: Well you can call me by name, Veert, or you can call me by value, Worth. That would have been sometime in the early 60s, when he was at Stanford. -Bill -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Thu, Nov 8, 2012 at 1:26 AM, Andrew Robinson andr...@r3dsolutions.com wrote: OK: Then copy by reference using map: values = zip( map( lambda:times, xrange(num_groups) ) ) if len(values) len(times) * num_groups ... Done. It's clearer than a list comprehension and you still really don't need a list multiply. That is not equivalent to the original. Even had you not omitted some parts: values = zip(samples, map(lambda i: times, range(num_groups))) This still has the problem that map returns a list of num_groups elements, each of which is times. The desired value to be passed into zip is a *single* sequence containing len(times) * num_groups elements. This is easily handled by list multiplication, but not so easily by map or by a single list comprehension. Looking back at the 'ini' solution you proposed before, I see that this also would be a problem there. Fixing the above, it would have to be something like: values = zip(samples, reduce(operator.add, map(lambda i: times, range(num_groups)), [])) Or from how I understand the 'ini' syntax to work: values = zip(samples, reduce(operator.add, [lambda: times, ini xrange(num_groups)], [])) Which brings to mind another point that I want to get to in a moment. But when I said that I would use map instead, I meant that *if* the body of the list comprehension is just a function application, then I would prefer to use map over the list comprehension. But in the above I see no benefit in using a lambda in the first place. Getting back to that other point, notice what we ended up doing in both of those constructions above: repeated list concatenation as a substitute for multiplication. In fact, when we multiply (aList * 5), this should be the equivalent of (aList + aList + aList + aList +aList), should it not? Clearly, however, there should be no implicit copying involved in mere list concatenation. For one thing, if the user wants to concatenate copies, that is quite easily done explicitly: (aList[:] + aList[:]) instead of (aList + aList). For another, list concatenation is less likely to be used for an initialization process. If list multiplication were to copy nested lists, then, this would break the intuitive notion that list multiplication is equivalent to repeated list concatenation. Yes, but you're very blind to history and code examples implementing the slice operation. slice usually depends on index; index does not depend on slice. Slice is suggested to be implemented by multiple calls to single indexes in traditional usage and documentation. ...and then by composing the elements located at those indexes into a subsequence. The xrange(,,)[:] implementation breaks the tradition, because it doesn't call index multiple times; nor does it return a result equivalent identical to doing that. Whether any given __getitem__ slicing implementation recursively calls __getitem__ with a series of indexes or not is an implementation detail. If it were possible to index a range object multiple times and then stuff the results into another range object, then the slicing result would be equivalent. The only reason it is not is that you cannot construct a range object in that fashion. I think that what you're expecting is that range(5)[:] should return a list in Python 3 because it returns a list in Python 2. This does not represent a change in slicing behavior -- in fact, all you got by slicing an xrange object in Python 2 was a TypeError. This represents an intentional break in backward compatibility between Python 2 and Python 3, which was the purpose of Python 3 -- to fix a lot of existing warts in Python by breaking them all at once, rather than progressively over a long string of versions. Users porting their scripts from Python 2 to Python 3 are advised to replace range(...) with list(range(...)) if what they actually want is a list, and I believe the 2to3 tool does this automatically. Once the range object is converted to a list, there is no further break with Python 2 -- slicing a list gives you a list, just as it always has. In a nutshell, yes: range(...)[:] produces a different result in Python 3 than in Python 2, just as it does without the slicing operation tacked on. It was never intended that scripts written for Python 2 should be able to run in Python 3 unchanged without careful attention to detail. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Monday, November 5, 2012 3:07:12 PM UTC+8, Chris Rebert wrote: On Sun, Nov 4, 2012 at 10:27 PM, Demian Brecht demianbre...@gmail.com wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 This is not clear in a name binding objective programming language. b=[1,2,3,4]*4 mb=[ b]*4 # check the behaviors and usages of reference copies # and shadow value copies and deep-value copies The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] (Obviously, I could have just hardcoded the initialization, but I'm too lazy to type all that out ;)) The behaviour I encountered seems a little contradictory to me. [None] * 4 creates four distinct elements in a single array while [[None] * 4] * 4 creates one distinct array of four distinct elements, with three references to it: Incorrect. In /both/ cases, the result is a list of length 4, whose elements are 4 (references to) the exact same object as the original list's element. Put simply, the list multiplication operator never copies objects; it just makes additional references to them. However, unlike a list object (as in your latter example), the object `None` is completely immutable (and what's more, a singleton value), so you just-so-happen *not to be able to* run into the same problem of mutating an object (assignment to an index of a list constitutes mutation of that list) that is referenced in multiple places, for you cannot mutate None in the first place!: x = None x.a = 42 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'NoneType' object has no attribute 'a' # it doesn't overload any mutating operators: type(None).__dict__.keys() ['__hash__', '__repr__', '__doc__'] # and it obviously has no instance variables, # so, we can't modify it in any way whatsoever! (Lists, on the other hand, define item assignment, .pop(), .remove(), and a few other mutator methods.) a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behavior Yes. It's also a FAQ: http://docs.python.org/2/faq/programming.html#how-do-i-create-a-multidimensional-list and if so, why? It's a general (albeit AFAIK unstated) principle that Python never copies objects unless you explicitly ask it to. You have encountered one example of this rule in action. In my mind either result makes sense, but the inconsistency is what throws me off. It is perfectly consistent, once you understand what list multiplication actually does. Cheers, Chris -- http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Fri, Nov 9, 2012 at 12:39 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x Can you enlighten us as to how this is better than either: x := -x or x := 0 - x ? I'm not seeing it. And I'm not seeing any nonnumeric that would benefit from being subtracted from itself twice (strings, arrays, sets, you can subtract them from one another but not usefully more than once). ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Fri, 09 Nov 2012 17:07:09 +1100, Chris Angelico wrote: On Fri, Nov 9, 2012 at 12:39 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x Can you enlighten us as to how this is better than either: x := -x or x := 0 - x ? I'm not seeing it. I'm hoping that Mark intended it as an example of crappy code he has spotted in some other language rather than a counter-example of something you would do. To be pedantic... there may very well be some (rare) cases where you actually do want x -= x rather than just x = 0. Consider the case where x could be an INF or NAN. Then x -= x should give x = NAN rather than zero. That may be desirable in some cases. At the very least, the compiler should NOT optimize away x = x - x to x = 0 if x could be a float, complex or Decimal. And I'm not seeing any nonnumeric that would benefit from being subtracted from itself twice (strings, arrays, sets, you can subtract them from one another but not usefully more than once). How do you subtract strings? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Fri, Nov 9, 2012 at 5:37 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
On Fri, 09 Nov 2012 17:07:09 +1100, Chris Angelico wrote:
Can you enlighten us as to how this is better than either:
x := -x
or
x := 0 - x
? I'm not seeing it.
I'm hoping that Mark intended it as an example of crappy code he has
spotted in some other language rather than a counter-example of something
you would do.
Ohh. Yeah, that figures. Huh.
To be pedantic... there may very well be some (rare) cases where you
actually do want x -= x rather than just x = 0. Consider the case where x
could be an INF or NAN. Then x -= x should give x = NAN rather than zero.
That may be desirable in some cases.
At the very least, the compiler should NOT optimize away x = x - x to
x = 0 if x could be a float, complex or Decimal.
Yep. In the specific case of integers, though, and in the specific
instance of CPU registers in assembly language, it's reasonable to
optimize it the *other* way - MOV reg,0 is a one-byte opcode and 1, 2,
or 4 bytes of immediate data, while SUB reg,reg (or XOR reg,reg) is a
two-byte operation regardless of data size. But that's
microoptimization that makes, uhh, itself-subtracted-from-itself sense
in Python.
And I'm not seeing any nonnumeric that would
benefit from being subtracted from itself twice (strings, arrays, sets,
you can subtract them from one another but not usefully more than once).
How do you subtract strings?
The same way you subtract sets. Same with arrays. Python doesn't do
either, but Python also doesn't do the := operator that the example
code demonstrated, so I didn't assume Python.
Pike v7.8 release 700 running Hilfe v3.5 (Incremental Pike Frontend)
Hello, world!-l;
(1) Result: Heo, word!
({1,2,3,3,2,3,1,2,1})-({2});
(2) Result: ({ /* 6 elements */
1,
3,
3,
3,
1,
1
})
Python spells it differently:
Hello, world!.replace(l,)
'Heo, word!'
Not sure how to do array subtraction other than with filter:
list(filter(lambda x: x!=2,[1,2,3,3,2,3,1,2,1]))
[1, 3, 3, 3, 1, 1]
But there's probably a way (list.remove only takes out the first
occurrence, so it's not equivalent).
In any case, subtracting something from _itself_ is only going to give
you an empty string, array, set, or whatever, and doing so a second
time is going to achieve nothing. Hence my comment.
But poor code we will always have with us, to paraphrase the Gospel of Matthew.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 09/11/2012 06:37, Steven D'Aprano wrote: On Fri, 09 Nov 2012 17:07:09 +1100, Chris Angelico wrote: On Fri, Nov 9, 2012 at 12:39 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 07/11/2012 01:55, Steven D'Aprano wrote: Who knows? Who cares? Nobody does: n -= n But I've seen this scattered through code: x := x - x - x Can you enlighten us as to how this is better than either: x := -x or x := 0 - x ? I'm not seeing it. I'm hoping that Mark intended it as an example of crappy code he has spotted in some other language rather than a counter-example of something you would do. Correct, CORAL 66 and pointed out to me by a colleague when another team member had resigned. To be pedantic... there may very well be some (rare) cases where you actually do want x -= x rather than just x = 0. Consider the case where x could be an INF or NAN. Then x -= x should give x = NAN rather than zero. That may be desirable in some cases. Interesting what comes up when we get chatting here. I hope we don't get punished for going off topic :) At the very least, the compiler should NOT optimize away x = x - x to x = 0 if x could be a float, complex or Decimal. X was an int so almost certainly optimised away by the SDL compiler on VMS of 1986 or 1987. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Steven D'Aprano writes: On Wed, 07 Nov 2012 00:23:44 +, MRAB wrote: I prefer the term reference semantics. Oh good, because what the world needs is yet another name for the same behaviour. - call by sharing - call by object sharing - call by object reference - call by object - call by value, where values are references (according to the Java community) - call by reference, where references refer to objects, not variables (according to the Ruby community) - reference semantics Anything else? http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing Something else: There's a call-by-* versus pass-by-* distinction, where the call-by-* would be rather different from any of the above: - call-by-value is what most languages now use: argument expressions are reduced to values before they are passed to the function / procedure / method / whatever. - call-by-name was something Algol 60 had by default: something like evaluating the argument expression every time its value is needed - call-by-need: argument expression is reduced to a value the first time its value is needed (if ever) - call-by-lazy (increasingly silly terminology, and I don't quite have an idea what it means in contrast to call-by-need) The modern confusions would then be mostly over the pass-by-* family, invariably using call-by-value in the above sense. The terminology for these tends to produce more heat than light, but I think the relevant distinctions are mostly just these: - can one modify the argument effectively [Python: yes] - can one modify the parameter with abandon [Python: don't] - can one swap [Python: no] - possibly: is it expensive to pass large objects? [Python: no] The actual rule in Scheme, Java, and Python is the same simple and sane rule: what are passed are values (argument expressions are fully evaluated before the actual call takes place), parameter passing does not involve any (observable) copying, and the arguments are bound to fresh variables (no aliasing of variables). Different communities use different words. Sometimes they use the same words about different things. Resulting in more heat than light :( (I'd have a few more things in the something-else department, but this is already much longer than I thought. Ends.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Le mercredi 7 novembre 2012 02:55:10 UTC+1, Steven D'Aprano a écrit : Two-dimensional arrays in Python using lists are quite rare. Anyone who is doing serious numeric work where they need 2D arrays is using numpy, not lists. There are millions of people using Python, so it's hardly surprising that once or twice a year some newbie trips over this. But it's not something that people tend to trip over again and again and again, like C's assignment is an expression misfeature. from vecmat6 import * from vmio5 import * Traceback (most recent call last): File eta last command, line 1, in module ImportError: No module named vmio5 from vmio6 import * from svdecomp6 import * mm = NewMat(3, 3) mm [[0.0, 0.0, 0.0], [0.0, 0.0, 0.0], [0.0, 0.0, 0.0]] mm[0][0] = 1.0; mm[0][1] = 2.0; mm[0][2] = 3.0 mm[1][0] = 11.0; mm[1][1] = 12.0; mm[1][2] = 13.0 mm[2][0] = 21.0; mm[2][1] = 22.0; mm[2][2] = 23.0 pr(mm, 'mm=') mm= ( 1.0e+000 2.0e+000 3.0e+000 ) ( 1.1e+001 1.2e+001 1.3e+001 ) ( 2.1e+001 2.2e+001 2.3e+001 ) aa, b, cc = SVDecomp(mm) pr(aa, 'aa=') aa= ( -8.08925e-002 -9.09280e-001 4.08248e-001 ) ( -4.77811e-001 -3.24083e-001 -8.16497e-001 ) ( -8.74730e-001 2.61114e-001 4.08248e-001 ) pr(b, 'b=') b= ( 4.35902e+001 1.37646e+000 1.93953e-016 ) pr(cc, 'cc=') cc= ( -5.43841e-001 7.33192e-001 4.08248e-001 ) ( -5.76726e-001 2.68499e-002 -8.16497e-001 ) ( -6.09610e-001 -6.79492e-001 4.08248e-001 ) bb = VecToDiagMat(b) cct = TransposeMat(cc) oo = MatMulMatMulMat(aa, bb, cct) pr(oo, 'aa * bb * cct=') aa * bb * cct= ( 1.0e+000 2.0e+000 3.0e+000 ) ( 1.1e+001 1.2e+001 1.3e+001 ) ( 2.1e+001 2.2e+001 2.3e+001 ) # or oo [[0.9991, 1.9993, 2.9982], [10.995, 11.99, 12.996], [20.986, 21.975, 22.986]] jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Nov 7, 2012 5:41 AM, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: If anything is to be done in this area, it would be better as an extension of list comprehensions, e.g. [[None times 5] times 10] which would be equivalent to [[None for _i in xrange(5)] for _j in xrange(10)] I think you're right that the meaning of list-int multiplication can't/shouldn't be changed if this way. A multidimensional list comprehension would be useful even for people who are using numpy as it's common to use a list comprehension to initialise a numpy array. A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 7 November 2012 11:11, Oscar Benjamin oscar.j.benja...@gmail.com wrote: On Nov 7, 2012 5:41 AM, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: If anything is to be done in this area, it would be better as an extension of list comprehensions, e.g. [[None times 5] times 10] which would be equivalent to [[None for _i in xrange(5)] for _j in xrange(10)] I think you're right that the meaning of list-int multiplication can't/shouldn't be changed if this way. A multidimensional list comprehension would be useful even for people who are using numpy as it's common to use a list comprehension to initialise a numpy array. A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] Hold on: why not just use multiplication? [0] * (2, 3) is an error now, and it makes total sense. Additionally, it's not breaking the no copy -- _ever_ rule because none of the lists existed before. The values inside the list would be by reference, as before, so lst * (x,) would be the same as lst * x if x is an integer. *I* would use this a lot. This is the first thing on this thread that makes a lot of sense to me. We do have to think of the potential problems, though. There are definitely some. For one, code that relies on lst * x throwing an error would break. It may confuse others - although I don't see how. But I don't see any big problems, so I really do like this idea. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 7 November 2012 13:39, Joshua Landau joshua.landau...@gmail.com wrote: On 7 November 2012 11:11, Oscar Benjamin oscar.j.benja...@gmail.com wrote: A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] Hold on: why not just use multiplication? [0] * (2, 3) is an error now, and it makes total sense. Additionally, it's not breaking the no copy -- _ever_ rule because none of the lists existed before. The values inside the list would be by reference, as before, so lst * (x,) would be the same as lst * x if x is an integer. The problem is that this operation is asymmetric. Currently int/list multiplication is commutative so that: ['a', 'b'] * 2 == 2 * ['a', 'b'] If you use this kind of multiplication what happens to the other cases? e.g. what do you give for: [0] * [2, 3] [2, 3] * [0] (2, 3) * [0] (2, 3) * (4, 5) and so on. Although Python does not guarantee commutativity of multiplication in general I think that since for lists it has always been commutative it would be bad to change that. Exponentiation is expected to be asymmetric and is currently unused so there is no ambiguity. The problem is if someone has already subclassed list and added an exponentiation method. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
After this post the only credibility you have left (with me, anyway) is that you seem to be willing to learn. So learn the way Python works before you try to reimplement it. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Oscar Benjamin wrote:
On Nov 7, 2012 5:41 AM, Gregory Ewing greg.ew...@canterbury.ac.nz
mailto:greg.ew...@canterbury.ac.nz wrote:
If anything is to be done in this area, it would be better
as an extension of list comprehensions, e.g.
[[None times 5] times 10]
which would be equivalent to
[[None for _i in xrange(5)] for _j in xrange(10)]
I think you're right that the meaning of list-int multiplication
can't/shouldn't be changed if this way.
A multidimensional list comprehension would be useful even for people
who are using numpy as it's common to use a list comprehension to
initialise a numpy array.
A more modest addition for the limited case described in this thread
could be to use exponentiation:
[0] ** (2, 3)
[[0, 0, 0], [0, 0, 0]]
What would happen with
-- [{}] ** (2, 3)
or
-- [my_custom_container()] ** (2, 3)
?
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
RE: Multi-dimensional list initialization
Gregory Ewing wrote: Roy Smith wrote: Call by social network? The called function likes the object. Depending on how it feels, it can also comment on some of the object's attributes. And then finds that it has inadvertently shared all its private data with other functions accessing the object. And this is where Dihedral (or whatever the bot is called) tells you that Python has no private variables. :) ~Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Nov 7, 2012 3:55 PM, Ethan Furman et...@stoneleaf.us wrote:
Oscar Benjamin wrote:
A more modest addition for the limited case described in this thread
could be to use exponentiation:
[0] ** (2, 3)
[[0, 0, 0], [0, 0, 0]]
What would happen with
-- [{}] ** (2, 3)
The list being exponentiated does nothing with its elements. The
exponentiation just tells it to create a list of distinct lists. In this
case each element of each sublist is the same dict. However if you assign
to an element of the sublist (rather than into the dict) it replaces the
dict in that sublist and not the others.
or
-- [my_custom_container()] ** (2, 3)
Ditto
Oscar
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 2012-11-07 05:05, Steven D'Aprano wrote: On Wed, 07 Nov 2012 00:23:44 +, MRAB wrote: Incorrect. Python uses what is commonly known as call-by-object, not call-by-value or call-by-reference. Passing the list by value would imply that the list is copied, and that appends or removes to the list inside the function would not affect the original list. This is not what Python does; the list inside the function and the list passed in are the same list. At the same time, the function does not have access to the original reference to the list and cannot reassign it by reassigning its own reference, so it is not call-by-reference semantics either. I prefer the term reference semantics. Oh good, because what the world needs is yet another name for the same behaviour. - call by sharing - call by object sharing - call by object reference - call by object - call by value, where values are references (according to the Java community) - call by reference, where references refer to objects, not variables (according to the Ruby community) - reference semantics Anything else? http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing The disadvantage of calling it call by ... is that it suggests that you're just talking about calling functions. What about binding in general, eg x = y? Does it make sense to still call it call by ...? -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Hi IAN!
On 11/06/2012 03:52 PM, Ian Kelly wrote:
On Tue, Nov 6, 2012 at 3:41 PM, Andrew Robinson
The objection is not nonsense; you've merely misconstrued it. If
[[1,2,3]] * 4 is expected to create a mutable matrix of 1s, 2s, and
3s, then one would expect [[{}]] * 4 to create a mutable matrix of
dicts. If the dicts are not copied, then this fails for the same
reason
:) The idea does create a multable list of dicts; just not a mutable
list of different dicts.
Q: How about if I use delegation to proxy a list?
A: Oh no, they definitely won't be copied.
Give an example usage of why someone would want to do this. Then we can
discuss it.
Seriously? Read a book on design patterns. You might start at SO:
http://stackoverflow.com/questions/832536/when-to-use-delegation-instead-of-inheritance
:) I wasn't discarding the argument, I was asking for a use case to
examine.
I know what a delegation *is*; but I'm not spending lots of times
thinking about this issue.
(Besides this thread just went more or less viral, and I can't keep up)
I have a book on design patterns -- in fact, the one called Design
Patterns by Gamma, Helm, Johnson, Vlissides. (Is it out of date
already or something?)
Please link to the objection being proposed to the developers, and
their reasoning for rejecting it. I think you are exaggerating.
From Google:
http://bugs.python.org/issue1408
http://bugs.python.org/issue12597
http://bugs.python.org/issue9108
http://bugs.python.org/issue7823
Note that in two out of these four cases, the reporter was trying to
multiply lists of dicts, not just lists of lists.
That's helpful. Thanks. I'll look into these.
Besides, 2D arrays are *not* rare and people *have* to copy internals of
them very often.
The copy speed will be the same or *faster*, and the typing less -- and the
psychological mistakes *less*, the elegance more.
List multiplication is not potentially useful for copying 2D lists,
only for initializing them. For copying an existing nested list,
you're still stuck with either copy.deepcopy() or a list
comprehension.
Yes, I totally agree.
But, as far as I know -- the primary use of list multiplication is
initialization.
That was my point about the most compact notation ought to be for the
most common case.
Initialization is a very common use case.
List comprehensions are appropriate for the other's.
Even D'Aprano thought the * operator was not a common operation; and I
suppose that when compared to other operations done in a program
(relative counting) he's correct; most programs are not primarily matrix
or initialization oriented.
It's hardly going to confuse anyone to say that lists are copied with list
multiplication, but the elements are not.
Every time someone passes a list to a function, they *know* that the list is
passed by value -- and the elements are passed by reference. People in
Python are USED to lists being the way to weird behavior that other
languages don't do.
Incorrect. Python uses what is commonly known as call-by-object, not
call-by-value or call-by-reference. Passing the list by value would
imply that the list is copied, and that appends or removes to the list
inside the function would not affect the original list.
Interesting, you avoided the main point lists are copied with list
multiplication.
But, in any event:
_Pass_ by value (not call by value) is a term stretching back 30 years;
eg: when I learned the meaning of the words. Rewording it as Call by
value is something that happened later, and the nuance is lost on those
without a very wide programming knowledge *and* age.
In any event:
All objects in Python are based on pointers; all parameters passed to
functions, etc, are *copies* of those pointers; (by pointer value).
I made the distinction between contents of the list and the list object
itself for that reason; I gave an explicit correction to the _pass_ by
value generalization by saying: (the elements are passed by reference).
The concept I gave, although archaically stated -- still correctly
represents what actually happens in Python and can be seen from it's
source code(s).
The point I am making is not generally true of everyone learning
Python; For some people obviously learn it from scratch. But, for
people who learn the language after a transition, this is a common FAQ;
how do I modify the variables by reference and not by value; -- the
answer is, you can't -- you must embed the return value in another
object; parameters are always passed the *same* way.
Every function written, then, has to decide when objects are passed to
it -- whether to modify or copy the object (internals) when modifying
it. That's all I meant.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, Nov 7, 2012 at 12:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: Interesting, you avoided the main point lists are copied with list multiplication. It seems that each post is longer than the last. If we each responded to every point made, this thread would fill a book. Anyway, your point was to suggest that people would not be confused by having list multiplication copy lists but not other objects, because passing lists into functions as parameters works in basically the same way. Except that it does not work the same way, because when lists are passed into functions, they are not copied at all. Nor are are any of their contents copied, lists or not. So actually I did address this point with the call-by-object tangent; I just did not explicitly link it back to your thesis. But, in any event: Pass by value (not call by value) is a term stretching back 30 years; eg: when I learned the meaning of the words. Rewording it as Call by value is something that happened later, and the nuance is lost on those without a very wide programming knowledge *and* age. Potayto, potahto. The distinction that you're describing is between strict versus non-strict evaluation strategies. Hinging the distinction on the non-descriptive words call and pass is lazy terminology that should never have been introduced in the first place. In any event: All objects in Python are based on pointers; all parameters passed to functions, etc, are *copies* of those pointers; (by pointer value). No, all parameters passed to functions are *objects*. Python itself has no concept of pointers. What you describe is true as an implementation detail for CPython but not necessarily true for other implementations, and not true at all for an abstract (implementation-independent) view of the language. I made the distinction between contents of the list and the list object itself for that reason; I gave an explicit correction to the pass by value generalization by saying: (the elements are passed by reference). The elements are not passed anywhere. Only the list object is passed to the function, which is completely agnostic of the fact that the list object happens to contain other objects. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 05:55 PM, Steven D'Aprano wrote: On Tue, 06 Nov 2012 14:41:24 -0800, Andrew Robinson wrote: Yes. But this isn't going to cost any more time than figuring out whether or not the list multiplication is going to cause quirks, itself. Human psychology *tends* (it's a FAQ!) to automatically assume the purpose of the list multiplication is to pre-allocate memory for the equivalent (using lists) of a multi-dimensional array. Note the OP even said 4d array. I'm not entirely sure what your point is here. The OP screwed up -- he didn't generate a 4-dimensional array. He generated a 2-dimensional array. If his intuition about the number of dimensions is so poor, why should his intuition about list multiplication be treated as sacrosanct? Yes he did screw up. There is a great deal of value in studying how people screw up, and designing interfaces which tend to discourage it. Candy machine interfaces. As they say, the only truly intuitive interface is the nipple. No it's not -- that interface really sucks. :) Have you ever seen a cat trying to suck a human nipple -- ? Or, have you ever asked a young child who was weaned early and doesn't remember nursing -- what a breast is for ? Once the oral stage is left, remaining behavior must be re-learned. There are many places where people's intuition about programming fail. And many places where Fred's intuition is the opposite of Barney's intuition. OK. But that doesn't mean that *all* places have opposite intuition; Nor does it mean that one intuition which is statistically *always* wrong shouldn't be discouraged, or re-routed into useful behavior. Take the candy machine, if the items being sold are listed by number -- and the prices are also numbers; it's very easy to type in the price instead of the object number because one *forgets* that the numbers have different meaning and the machine can't always tell from the price, which object a person wanted (duplicate prices...); Hence a common mistake... people get the wrong item, by typing in the price. By merely avoiding a numeric keypad -- the user is re-routed into choosing the correct item by not being able to make the mistake. For this reason, Python tends to *like* things such as named parameters and occasionally enforces their use. etc. Even more exciting, there are places where people's intuition is *inconsistent*, where they expect a line of code to behave differently depending on their intention, rather than on the code. And intuition is often sub-optimal: e.g. isn't it intuitively obvious that 42 + 1 should give 43? (Unless it is intuitively obvious that it should give 421.) I agree, and in places where an *exception* can be raised; it's appropriate to do so. Ambiguity, like the candy machine, is *bad*. So while I prefer intuitively obvious behaviour where possible, it is not the holy grail, and I am quite happy to give it up. where possible; OK, fine -- I agree. I'm not happy to give it up; but I am willing. I don't like the man hours wasted on ambiguous behavior; and I don't ever think that should make someone happy. The OP's original construction was simple, elegant, easy to read and very commonly done by newbies learning the language because it's *intuitive*. His second try was still intuitive, but less easy to read, and not as elegant. Yes. And list multiplication is one of those areas where intuition is suboptimal -- it produces a worse outcome overall, even if one minor use- case gets a better outcome. I'm not disputing that [[0]*n]*m is intuitively obvious and easy. I'm disputing that this matters. Python would be worse off if list multiplication behaved intuitively. How would it be worse off? I can agree, for example, that in C -- realloc -- is too general. One can't look at the line where realloc is being used, and decide if it is: 1) mallocing 2) deleting 3) resizing Number (3) is the only non-redundant behavior the function provides. There is, perhaps, a very clear reason that I haven't discovered why the extra functionality in list multiplication would be bad; That reason is *not* because list multiplication is unable to solve all the copying problems in the word; (realloc is bad, precisely because of that); But a function ought to do at least *one* thing well. Draw up some use cases for the multiplication operator (I'm calling on your experience, let's not trust mine, right?); What are all the Typical ways people *Do* to use it now? If those use cases do not *primarily* center around *wanting* an effect explicitly caused by reference duplication -- then it may be better to abolish list multiplication all together; and rather, improve the list comprehensions to overcome the memory, clarity, and speed pitfalls in the most common case of initializing a list. For example, in initialization use cases; often the variable of a for loop isn't needed and all the initializers have parameters which only need to be
Re: Multi-dimensional list initialization
*Spoiler:* You've convinced me. On 7 November 2012 14:00, Oscar Benjamin oscar.j.benja...@gmail.com wrote: On 7 November 2012 13:39, Joshua Landau joshua.landau...@gmail.com wrote: On 7 November 2012 11:11, Oscar Benjamin oscar.j.benja...@gmail.com wrote: A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] Hold on: why not just use multiplication? [0] * (2, 3) is an error now, and it makes total sense. Additionally, it's not breaking the no copy -- _ever_ rule because none of the lists existed before. The values inside the list would be by reference, as before, so lst * (x,) would be the same as lst * x if x is an integer. The problem is that this operation is asymmetric. Currently int/list multiplication is commutative so that: ['a', 'b'] * 2 == 2 * ['a', 'b'] I see. I agree that that is a valid point. Remember, though, that we could just keep this behaviour: [0] * (2, 3) == (2, 3) * [0] If you use this kind of multiplication what happens to the other cases? e.g. what do you give for: [0] * [2, 3] Nothing. If you allowed lists to multiply this time, why not with your suggestion? We should require a tuple and a list. [2, 3] * [0] Same. (2, 3) * [0] == [0] * (2, 3) (2, 3) * (4, 5) Nothing. and so on. Although Python does not guarantee commutativity of multiplication in general I think that since for lists it has always been commutative it would be bad to change that. Agreed Exponentiation is expected to be asymmetric and is currently unused so there is no ambiguity. The problem is if someone has already subclassed list and added an exponentiation method. How is that a problem? They just wont get the functionality. That said, losing: [0] * (2, 3) == [0] * [2, 3] would mean losing duck-typing in general. *Thus*, I fully agree with your choice of exponentiation. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 10:56 PM, Demian Brecht wrote: My question was *not* based on what I perceive to be intuitive (although most of this thread has now seemed to devolve into that and become more of a philosophical debate), but was based on what I thought may have been inconsistent behaviour (which was quickly cleared up with None being immutable and causing it to *seem* that the behaviour was inconsistent to the forgetful mind). I originally brought up intuitive; and I don't consider the word to mean an exclusive BEST way -- I meant it to mean easily guessed or understood. An intelligent person can see when there may be more than one reasonable explanation -- ergo: I just called your OP intelligent, even if you were wrong; and D'Aprano ripped you for being wrong. The debate is degenerating because people are _subjectively_ judging other people's intelligence. The less intelligent a person is, the more black and white their judgements _tend_ to be. As you touch on here, intuition is entirely subjective. If you're coming from a C/C++ background, I'd think that your intuition would be that everything's passed by value unless explicitly stated. Yup -- that's my achillies heel and bias, I'm afraid. I learned basic, then assembly, and then pascal, and then fortran77 with C (historically in that order) In my view, pass by value vs. reference always exists at the hardware/CPU level regarless of the language; and regardless of whether the language hides the implementation details or not; I'm an EE; I took software engineering to understand the clients who use my hardware, and to make my hardware drivers understandable to them by good programming practices. An EE's perspective often lead to doing efficient things which are hard to understand; That's why I look for a consensus (not a compromise) before implementing speed/memory improvements and ways to clarify what is being done. Someone coming from another background (Lua perhaps?) would likely have entirely different intuition. Yes, they might be ignorant of what LUA is doing at the hardware level; even though it *is* doing it. So while I prefer intuitively obvious behaviour where possible, it is not the holy grail, and I am quite happy to give it up. I fail to see where there has been any giving up on intuitiveness in the context of this particular topic. In my mind, intuitiveness is generally born of repetitiveness and consistency. YES I think a good synonym would be habit; and when a habit is good -- it's called strength, or virtue; When it's bad it's called vice or sin or bad programming habit. :) Virtues don't waste people's time in debugging. As everything in Python is a reference, it would seem to me to be inconsistent to treat expressions such as [[obj]*4]*4 un-semantically (Pythonically speaking) and making it *less* intuitive. I agree that Python would definitely be worse off. That's a fair opinion. I was pleasantly surprised when the third poster actually answered the WHY question with the idea that Python always copies by reference unless forced to do deep copy. That's intuitive, and as a habit (not a requirement) Python implements things that way. I've already raised the question about why one would want a multiplier at all, if it were found that the main desired use case never *wants* all objects to change together. I laid out a potential modification of list comprensions; which, BTW, copy by re-instantiating rather than reference; so the paradigm of Python is wrong in that case But, I think the modifications in that context can't be argued against as easily as list multiplication (For the same reason that comprehensions already break the copy by reference mold ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 07/11/2012 22:02, Andrew Robinson wrote: You're doing extremely well, you've overtaken Xah Lee as the biggest waste of space on this list. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, 07 Nov 2012 17:17:02 +, MRAB wrote: The disadvantage of calling it call by ... is that it suggests that you're just talking about calling functions. *shrug* There are already two synonyms for this, call by ... and pass by They are old, venerable terms dating back to Algol and possibly even older. All the way back to Fortran perhaps? What about binding in general, eg x = y? Does it make sense to still call it call by ...? Sure, why not? The person who prepares beef tartare or sushimi is still called the cook. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 7 November 2012 22:16, Joshua Landau joshua.landau...@gmail.com wrote: On 7 November 2012 14:00, Oscar Benjamin oscar.j.benja...@gmail.com wrote: On 7 November 2012 13:39, Joshua Landau joshua.landau...@gmail.com wrote: On 7 November 2012 11:11, Oscar Benjamin oscar.j.benja...@gmail.com wrote: A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] Exponentiation is expected to be asymmetric and is currently unused so there is no ambiguity. The problem is if someone has already subclassed list and added an exponentiation method. How is that a problem? They just wont get the functionality. This is absolutely contrived but: Library A defines a subclass of list that adds an exponentiation operator thinking that it's okay to still use these objects as lists. Library B has an API that expects a list and tries to use the list copy-exponentiation on its input. A user passes a list type object from library A into library B and hopefully gets an error but possibly gets a subtle bug that is hard to track down. It doesn't sound plausible to me but at least in principle there is a backward compatibility problem. That said, losing: [0] * (2, 3) == [0] * [2, 3] would mean losing duck-typing in general. *Thus*, I fully agree with your choice of exponentiation. Also there's no reason why tuples couldn't have the same exponentiation operator (although for them it would be no different from repeated multiplication). Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, Nov 7, 2012 at 3:02 PM, Andrew Robinson
andr...@r3dsolutions.com wrote:
Draw up some use cases for the multiplication operator (I'm calling on your
experience, let's not trust mine, right?); What are all the Typical ways
people *Do* to use it now?
If those use cases do not *primarily* center around *wanting* an effect
explicitly caused by reference duplication -- then it may be better to
abolish list multiplication all together; and rather, improve the list
comprehensions to overcome the memory, clarity, and speed pitfalls in the
most common case of initializing a list.
Why? Just to get rid of an FAQ?
Here's one of the more interesting uses from my own code:
values = zip(samples, times * num_groups)
if len(values) len(times) * num_groups:
# raise an error
Converting that multiplication to a generator expression would look like this:
values = zip(samples, (t for _ in range(num_groups) for t in times))
That's not particularly hairy, but I do assert that it is
substantially less readable, and more so because it loses the symmetry
with the following if condition.
The recipes in the itertools docs also include this example, which
notably depends on the list containing multiple references to the same
iterator:
def grouper(n, iterable, fillvalue=None):
Collect data into fixed-length chunks or blocks
# grouper(3, 'ABCDEFG', 'x') -- ABC DEF Gxx
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)
Replacing the list multiplication in that function with a list
comprehension would be awkward, as the obvious replacement of
[iter(iterable) for _ in range(n)] would produce different results.
For example, in initialization use cases; often the variable of a for loop
isn't needed and all the initializers have parameters which only need to be
evaluated *once* (no side effects).
Hence, there is an opportunity for speed and memory gains,while maintaining
clarity and *consistency*.
Some ideas of use cases:
[ (0) in xrange(10) ] # The function to create a tuple cache's the
parameter '0', makes 10 (0)'s
[ dict.__new__(dict) in xrange(10) ] # dict.__new__, The dict parameter is
cached -- makes 10 dicts.
[ lambda x:(0) in xrange(10) ] # lambda caches (0), returns a *reference* to
it multiple times.
How exactly do you propose to indicate to the compiler which parts of
the expressions are meant to be cached, and which are not?
Bull. Even in the last thread I noted the range() object produces
special cases.
range(0,5)[1]
1
range(0,5)[1:3]
range(1, 3)
What's the special case here? What do you think is copied?
You take a slice of a range object, you get a new range object.
You were'nt paying attention, OCCASIONALLY, get an integer, or a list.
range(3)[2]
2
LK! That's not a range object, that's an integer. Use Python 3.2 and
try it.
Of course you got an integer. You took an index of the range object,
not a slice. The rule is that taking an index of a sequence returns
an element; taking a slice of a sequence returns a sub-sequence. You
still have not shown any inconsistency here.
Game programmers routinely use 2D lists to represent the screen layout;
For example, they might use 'b' to represent a brick tile, and 'w' to
represent a water tile.
In many cases it may be simpler to use a plain list of strings:
screen = [
s,
ssbss,
sbbbs,
b,
]
py x = [{}]*5
py x
[{}, {}, {}, {}, {}]
No, I showed what happed when you do {}*3;
That *DOESN'T* work; You aren't multiplying the dictionary, you are
multiplying the LIST of dictionaries. Very different things.
You were complaining that my method doesn't multiply them -- well, gee --
either mine DOES or python DOESN'T. Double standards are *crap*.
No, he wasn't. He was talking about multiplying lists of dicts, and
whether the dicts are then copied or not, just like every other QA
item in that dialogue was concerning whether item X in a list should
expect to be copied when the containing list is multiplied.
You are the only one talking about applying the multiplication
operator to dicts.
Huh?
I'm not yelling any more than you are. Are ???YOU??? yelling?
Perhaps you're not aware that on the Internet, TYPING IN ALL CAPS is
commonly construed as SHOUTING.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/07/2012 05:39 AM, Joshua Landau wrote: On 7 November 2012 11:11, Oscar Benjamin wrote: On Nov 7, 2012 5:41 AM, Gregory Ewing wrote: If anything is to be done in this area, it would be better as an extension of list comprehensions, e.g. [[None times 5] times 10] which would be equivalent to [[None for _i in xrange(5)] for _j in xrange(10)] Oscar, I'm really in agreement with you; I think that it's better to group all *special* array/list constructions into a single logical unit which will show up in the same part of the Python documentation. A multidimensional list comprehension would be useful even for people who are using numpy as it's common to use a list comprehension to initialise a numpy array. I hadn't paid that much attention; but I think that's true of people using the newer releases of Numpy. A Very interesting point... Thank you for mentioning it. A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] I'm against over using the math operators, for the reason that matrix and vector algebra have meanings mathematicians desire (rightly) to maintain. Numpy users might find matricies overloaded to do these things in the future -- and then it becomes unclear whether an initialization is happening or a mathematical operation. I think it best just not to set up an accident waiting to happen in the first place. Hold on: why not just use multiplication? [0] * (2, 3) Would you consider that better than [0].nest(2).nest(3) ? or [0].nest(2,3) ? (I'm against multiplication, but I'm still interested in what you find attractive about it.) We do have to think of the potential problems, though. There are definitely some. For one, code that relies on lst * x throwing an error would break. It may confuse others - although I don't see how. Excellent observation: People relying on an exception, would be in the try: operation. So, since lst * int does not cause an exception; they would need a reason to be concerned that someone passed in a list instead of an integer. Semantically, the same KIND of result happens, lst is in some way duplicated; so if the result is accepted, it likely would work in place of an integer. So, the concern would be where someone wanted to detect the difference between an integer and a list, so as to run some alternate algorithm. Eg, say a vector multiply, or similar operation. The design would want to shadow * and call a method to do the multiply; You'd have a fragment possibly like the following: ... try: ret = map( lambda x: x*rightSide, leftSide ) except TypeError: for i in rightSide: self.__mul__( rightSide, i ) # recursive call to __mul__ ... That's a common technique for type checking dating from earlier releases of Python, where the type attribute wasn't available. It also works based on functionality, not specific type -- so objects which work alike (subclasses, alternate reinventions of the wheel) also can be handled. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Andrew, it appears that your posts are being eaten or rejected by my ISP's news server, because they aren't showing up for me. Possibly a side- effect of your dates being in the distant past? So if you have replied to any of my posts, I haven't seen them. In any case, I wanted to ask a question: On Wed, 07 Nov 2012 14:01:19 -0700, Ian Kelly wrote: On Wed, Nov 7, 2012 at 12:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: [...] But, in any event: Pass by value (not call by value) is a term stretching back 30 years; eg: when I learned the meaning of the words. Rewording it as Call by value is something that happened later, and the nuance is lost on those without a very wide programming knowledge *and* age. Every now and again I come across somebody who tries to distinguish between call by foo and pass by foo, but nobody has been able to explain the difference (if any) to me. When you CALL a function, you PASS values to it. Hence the two terms are effectively synonyms, and both refer to the evaluation strategy when binding arguments to parameters. If you believe that is incorrect, can you point me to something explaining the difference? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/07/2012 01:01 PM, Ian Kelly wrote: On Wed, Nov 7, 2012 at 12:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: Interesting, you avoided the main point lists are copied with list multiplication. It seems that each post is longer than the last. If we each responded to every point made, this thread would fill a book. It already is :) Anyway, your point was to suggest that people would not be confused by having list multiplication copy lists but not other objects, because passing lists into functions as parameters works in basically the same way. Not quite; Although I wasn't clear; The variable passed in is by *value* in contradistinction to the list which is by reference. Python does NOT always default copy by reference *when it could*; that's the point. Hence the programmer has to remember in foo( x,y ), the names x and y when assigned to -- *DONT* affect the variables from which they came. But any object internals do affect the objects everywhere. A single exception exists; My thesis is for a single exception as well -- I think Python allows that kind of thinking. So actually I did address this point with the call-by-object tangent; I just did not explicitly link it back to your thesis. My apology for not proof reading my statements for clarity. It was definitely time for a nap back then. Potayto, potahto. The distinction that you're describing is between strict versus non-strict evaluation strategies. Hinging the distinction on the non-descriptive words call and pass is lazy terminology that should never have been introduced in the first place. I would do it again. Other's have already begun to discuss terminology with you -- I won't double team you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 8 November 2012 00:00, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Andrew, it appears that your posts are being eaten or rejected by my ISP's news server, because they aren't showing up for me. Possibly a side- effect of your dates being in the distant past? So if you have replied to any of my posts, I haven't seen them. In any case, I wanted to ask a question: On Wed, 07 Nov 2012 14:01:19 -0700, Ian Kelly wrote: On Wed, Nov 7, 2012 at 12:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: [...] But, in any event: Pass by value (not call by value) is a term stretching back 30 years; eg: when I learned the meaning of the words. Rewording it as Call by value is something that happened later, and the nuance is lost on those without a very wide programming knowledge *and* age. Every now and again I come across somebody who tries to distinguish between call by foo and pass by foo, but nobody has been able to explain the difference (if any) to me. When you CALL a function, you PASS values to it. Hence the two terms are effectively synonyms, and both refer to the evaluation strategy when binding arguments to parameters. If you believe that is incorrect, can you point me to something explaining the difference? Did you also miss MRAB's post above? It made sense to me. MRAB wrote: The disadvantage of calling it call by ... is that it suggests that you're just talking about calling functions. What about binding in general, eg x = y? Does it make sense to still call it call by ...? -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 7 November 2012 23:55, Andrew Robinson andr...@r3dsolutions.com wrote: On 11/07/2012 05:39 AM, Joshua Landau wrote: A more modest addition for the limited case described in this thread could be to use exponentiation: [0] ** (2, 3) [[0, 0, 0], [0, 0, 0]] I'm against over using the math operators, for the reason that matrix and vector algebra have meanings mathematicians desire (rightly) to maintain. Numpy users might find matricies overloaded to do these things in the future -- and then it becomes unclear whether an initialization is happening or a mathematical operation. I think it best just not to set up an accident waiting to happen in the first place. I'm going to say right now that I'm very much fond of the exponentiation proposal. Multiplication on Numpy arrays is already completely disjoint to multiplication on lists, and that is probably completely disjoint to all sorts of mathematical meanings. I don't personally feel that anyone who knows what [0] * 3 is would *assume* (although they may suppose) that exponentiation will be a maths operator. When I saw [0] ** (2, 3), I knew what it did before I read anything else. I know I had the context of the posts above, so it isn't a fair comparison, but it seems really obvious an extension. It's so closely linked to * (if not for the ambiguities, I would have preferred multiplication) that it makes total sense. Even if you think of 4 ** 5 as 4 * 4, 5 times, there is a direct mental link to what is happening. Hold on: why not just use multiplication? [0] * (2, 3) Would you consider that better than [0].nest(2).nest(3) ? or [0].nest(2,3) ? (I'm against multiplication, but I'm still interested in what you find attractive about it.) Yes. Having [0] * 2 with a distinct but fundamentally the same (it's just gotten extra dimensions) partner that is called in a very similar way is a good thing. I'd feel equally unhappy with 4 * 3 partnering with (4).pow(3)* as I would with your .nest(2, 3) and I like the iterated ones even less because I don't see it as obviously possible for them to even work. [0].nest(2) - [[0], [0]] ? [[0], [0]].nest(3) - [[0,0,0], [0,0,0]] ??? (what about 3d?) * Even if you could write that as 4.pow(3) because floats didn't exist or something. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 08/11/12 12:06, Oscar Benjamin wrote: On 7 November 2012 22:16, Joshua Landaujoshua.landau...@gmail.com wrote: That said, losing: [0] * (2, 3) == [0] * [2, 3] would mean losing duck-typing in general. There are precedents for this kind of thing; the string % operator treats tuples specially, for example. I don't think it's all that bad if you regard the tuple as effectively part of the syntax. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/07/2012 03:39 PM, Ian Kelly wrote: Why? Just to get rid of an FAQ? :-) Here's one of the more interesting uses from my own code: OK, and is this a main use case? (I'm not saying it isn't I'm asking.) Replacing the list multiplication in that function with a list comprehension would be awkward, as the obvious replacement of [iter(iterable) for _ in range(n)] would produce different results. Yes. I have a thought on that. How exactly do you propose to indicate to the compiler which parts of the expressions are meant to be cached, and which are not? Exactly? OK; Here's what I would consider a safe implementation -- but it could be improved later. There is a special keyword which signals the new type of comprehension; A normal comprehension would say eg: '[ foo for i in xrange ]'; but when the 'for i in' is reduced to a specific keyword such as 'ini' (instead of problematic 'in') the caching form of list comprehension would start. So, then, just like a comprehension -- the interpreter will begin to evaluate the code from the opening bracket '['; But anything other than a function/method will raise a type error (people might want to change that, but it's safe). The interpreter then caches all functions/initialiser methods it comes into contact with. Since every function/method has a parameter list (even if empty); The interpreter would evaluate the parameter list on the first pass through the comprehension, and cache each parameter list with it's respective function. When the 'ini' keyword is parsed a second time, Python would then evaluate each cached function on its cached parameter list; and the result would be stored in the created list. This cached execution would be repeated as many times as is needed. Now, for your example: values = zip(samples, times * num_groups) if len(values) len(times) * num_groups: # raise an error Might be done with: values = zip( samples, [ lambda:times, ini xrange(num_groups) ] ) if len(values) len(times) * num_groups The comma after the lambda is questionable, and this construction would be slower since lambda automatically invokes the interpreter; but it's correct. If you could provide a built in which returns a reference to the parameter passed to it; that would run at max system speed; by default, all built-in object initializers are maximally fast. The key difference is that the ini syntax evaluates the parameter lists only once; and the ini's purpose is for repeating an initialization of the same kind of object in multiple different places. As an aside, how would you do the lambda inside a list comprehension? [lambda:6 for i in xrange(10) ] # Nope. Generic lists allow a spurrious comma, so that [ 3,3,3, ] = [3,3,3] dropped; [lambda:6, for i in xrange(10) ] # but this is no good. I have to do: def ref(): return 6 [ref(x) for i in xrange(10) ] Of course you got an integer. You took an index of the range object, not a slice. The rule is that taking an index of a sequence returns an element; taking a slice of a sequence returns a sub-sequence. You still have not shown any inconsistency here. Because it's an arbitrary rule which operates differently than the traditional idea shown in python docs? slice.indices() is *for* (QUOTE)representing the _set of indices_ specified by _range_(start, stop, step) http://docs.python.org/2/library/functions.html#slice There are examples of python doing this; use Google... They use slice indices() to convert negative indexes into positive ones _compatible with range()_. some_getitem_method_in_a_subclass_foo( self, range ): ret=[] for i in xrange( range.indices( len(self) ) ): ret.append( self.thing[i] ) return ret The return is equivalent to a range object in the sense that it is an iterator object, but it's not the same iterator object. It will still work with legacy code since different iterators can be interchanged so long as they return the same values. No, he wasn't. He was talking about multiplying lists of dicts, and whether the dicts are then copied or not, just like every other QA item in that dialogue was concerning whether item X in a list should expect to be copied when the containing list is multiplied. I already told him several times before that what the answer was; It doesn't copy anything except the list itself. Then he asks, does it multiply dicts and no mention of it being inside a list. He's browbeating a dead horse. Perhaps you're not aware that on the Internet, TYPING IN ALL CAPS is commonly construed as SHOUTING. Sure, and people say: THIS IS YELLING, AND I AM DOING IT HERE AS AN EXAMPLE. This is STRESS. This is SHOCK! I don't recall typing any _full sentence_ in all caps, if I did, I'm awfully sorry. I didn't mean it. Yes, he is beginning to get condescendingly exasperating. Everyone else seems to understand 85+% of what I say, correctly. He doesn't; and now
Re: Multi-dimensional list initialization
On Thu, 08 Nov 2012 00:30:53 +, Oscar Benjamin wrote: Every now and again I come across somebody who tries to distinguish between call by foo and pass by foo, but nobody has been able to explain the difference (if any) to me. When you CALL a function, you PASS values to it. Hence the two terms are effectively synonyms, and both refer to the evaluation strategy when binding arguments to parameters. If you believe that is incorrect, can you point me to something explaining the difference? Did you also miss MRAB's post above? It made sense to me. You mean MRABs post which I replied to? Yes, I must have missed it :-P But seriously, no I didn't miss it. He doesn't give any evidence that there is a difference between call by ... and pass by ... when talking about binding arguments to formal parameters. His objection to call by ... is that it doesn't make it clear that the evaluation rules apply to simple binding/assignment as well as calling functions. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, 07 Nov 2012 16:24:22 -0800, Andrew Robinson wrote: On 11/07/2012 01:01 PM, Ian Kelly wrote: [...] Anyway, your point was to suggest that people would not be confused by having list multiplication copy lists but not other objects, because passing lists into functions as parameters works in basically the same way. Not quite; Although I wasn't clear; The variable passed in is by *value* in contradistinction to the list which is by reference. Python does NOT always default copy by reference *when it could*; that's the point. It isn't clear to me whether you are describing what you think Python *actually* does, versus what you wish it *would* do, or what it *could* do in some abstract hypothetical sense. It certainly is not true that Python passes the variable by value, and lists by reference. Arguments are not passed to functions either by value or by reference. There is a trivial test for pass-by-value semantics: does the value get copied? We can see that Python does not copy arguments: py def test(x): ... print id(x) ... py spam = [] py print id(spam); test(spam) 3071264556 3071264556 The argument is not copied, therefore Python is not pass-by-value. There is also an easy test for pass-by-reference semantics: can you write a procedure which, given two variables, swaps the contents of the variables? In Pascal, that is trivial. procedure swap(var a: int, var b: int): var tmp: int; begin tmp := a; a := b; b := a; end; swap(x, y); (if I've remembered my Pascal syntax correctly). In Python, you can swap two values like this: a, b = b, a but that's not sufficient. The test is to do the swap inside a function: def swap(a, b): return b, a b, a = swap(a, b) But that fails too, since the assignment is still taking place outside the function. It turns out that there is no way in Python to write such a swap function. Tricks such as passing the variable names as strings, then using exec, are hacks and don't count. Python is not pass by reference either. Hence the programmer has to remember in foo( x,y ), the names x and y when assigned to -- *DONT* affect the variables from which they came. But any object internals do affect the objects everywhere. Ummm yes? The programmer has to remember Python's execution model in order to correctly predict what Python will do. What's your point? A single exception exists; There is no such exception in Python. Python always uses the same argument passing (parameter binding) semantics. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/07/2012 04:00 PM, Steven D'Aprano wrote: Andrew, it appears that your posts are being eaten or rejected by my ISP's news server, because they aren't showing up for me. Possibly a side- effect of your dates being in the distant past? Date has been corrected since two days ago. It will remain until a reboot Ignorance, though, might be bliss... Every now and again I come across somebody who tries to distinguish between call by foo and pass by foo, but nobody has been able to explain the difference (if any) to me. I think the Call by foo came into vogue around the time of C++; Eg: It's in books like C++ for C programmers; I never saw it used before then so I *really* don't know for sure... I know Pass by value existed all the way back to the 1960's. I see pass by in my professional books from those times and even most newer ones; but I only find Call by value in popular programming books of more recent times. (Just my experience) So -- I guess the reason is that when invoking a subroutine, early hardware often had an assembler mnemonic by the name call. See for example: Intelx86 hardware books from the 1970's; Most early processors (like the MC6809E, and 8080) allow both direct and indirect *references* to a function (C would call them function pointers); So, occasionally early assembly programs comment things like: ; dynamic VESA libraries are called by value in register D.; And they meant that register D is storing a function call address from two or more vesa cards. It had little to do with the function's parameters, (which might be globals anyway) (It procedural dynamic binding!) Today, I don't know for sure -- so I just don't use it. pass indicates a parameter of the present call; but not the present call itself. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, Nov 7, 2012 at 8:13 PM, Andrew Robinson andr...@r3dsolutions.com wrote: OK, and is this a main use case? (I'm not saying it isn't I'm asking.) I have no idea what is a main use case. There is a special keyword which signals the new type of comprehension; A normal comprehension would say eg: '[ foo for i in xrange ]'; but when the 'for i in' is reduced to a specific keyword such as 'ini' (instead of problematic 'in') the caching form of list comprehension would start. FYI, the Python devs are not very fond of adding new keywords. Any time a new keyword is added, existing code that uses that word as a name is broken. 'ini' is particularly bad, because 1) it's not a word, and 2) it's the name of a common type of configuration file and is probably frequently used as a variable name in relation to such files. So, then, just like a comprehension -- the interpreter will begin to evaluate the code from the opening bracket '['; But anything other than a function/method will raise a type error (people might want to change that, but it's safe). The interpreter then caches all functions/initialiser methods it comes into contact with. Since every function/method has a parameter list (even if empty); The interpreter would evaluate the parameter list on the first pass through the comprehension, and cache each parameter list with it's respective function. When the 'ini' keyword is parsed a second time, Python would then evaluate each cached function on its cached parameter list; and the result would be stored in the created list. This cached execution would be repeated as many times as is needed. Now, for your example: values = zip(samples, times * num_groups) if len(values) len(times) * num_groups: # raise an error Might be done with: values = zip( samples, [ lambda:times, ini xrange(num_groups) ] ) if len(values) len(times) * num_groups The comma after the lambda is questionable, and this construction would be slower since lambda automatically invokes the interpreter; but it's correct. How is this any better than the ordinary list comprehension I already suggested as a replacement? For that matter, how is this any better than list multiplication? Your basic complaint about list multiplication as I understand it is that the non-copying semantics are unintuitive. Well, the above is even less intuitive. It is excessively complicated and almost completely opaque. If I were to come across it outside the context of this thread, I would have no idea what it is meant to be doing. As an aside, how would you do the lambda inside a list comprehension? As a general rule, I wouldn't. I would use map instead. [lambda:6 for i in xrange(10) ] # Nope. Thak constructs a list of 10 functions and never calls them. If you want to actually call the lambda, then: [(lambda: 6)() for i in range(10)] or: map(lambda i: 6, range(10)) But note that the former creates equivalent 10 functions and calls each of them once, whereas the latter creates one function and calls it ten times. Of course you got an integer. You took an index of the range object, not a slice. The rule is that taking an index of a sequence returns an element; taking a slice of a sequence returns a sub-sequence. You still have not shown any inconsistency here. Because it's an arbitrary rule which operates differently than the traditional idea shown in python docs? slice.indices() is *for* (QUOTE)representing the set of indices specified by range(start, stop, step) http://docs.python.org/2/library/functions.html#slice slice.indices() has nothing to do with it. Indexing a sequence and calling the .indices() method on a slice are entirely different operations. The slice.indices method is a utility method meant to be called by __getitem__ implementations when doing slicing, not an implementation of indexing. When a sequence is indexed, there is no slice. That method is not related in any way to the semantics of indexing a sequence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/05/2012 10:07 PM, Chris Angelico wrote: On Tue, Nov 6, 2012 at 4:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: I really don't think doing a shallow copy of lists would break anyone's program. Well, it's a change, a semantic change. It's almost certainly going to break _something_. But for the sake of argument, we can suppose that the change could be made. Would it be the right thing to do? Shallow copying by default would result in extremely weird behaviour. All the same confusion would result, only instead of comparing [None]*4 with [[None]]*4, there'd be confusion over the difference between [[None]]*4 and [[[None]]]*4. I don't think it would help anything, and it'd result in a lot more work for no benefit. ChrisA I don't follow. a=[ None ]*4 would give a=[ None, None, None, None ] as usual. All four None's would be the same object, but there are automatically 4 different pointers to it. Hence, a[0]=1 would give a=[ 1, None, None, None ] as usual. a=[ [None] ]*4 would give a=[ [None], [None], [None], [None] ] as usual BUT: a[0][0] = 1 would no longer give a=[ [1],[1],[1],[1] ] *Rather* it would give a=[ [1].[None].[None],[None] ] The None objects are all still the same one, BUT the lists themselves are different. Again, a=[ [alpha,beta] * 4 ] would give: a=[ [alpha,beta], [alpha,beta], [alpha,beta], [alpha,beta] ] All four strings, alpha, are the same object -- but there are 5 different lists; The pointers inside the initial list are copied four times -- not the string objects; But the *lists* themselves are created new for each replication. If you nest it another time; [[[None]]]*4, the same would happen; all lists would be independent -- but the objects which aren't lists would be refrenced-- not copied. a=[[[alpha,beta]]]*4 would yield: a=[[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']]] and a[0][0]=1 would give [[1],[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta' rather than a=[[1], [1], [1], [1]] Or at another level down: a[0][0][0]=1 would give: a=[[[1, 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']] ] rather than a=[[[1, 'beta']], [[1, 'beta']], [[1, 'beta']], [[1, 'beta']]] The point is, there would be no difference at all noticed in what data is found where in the array; the *only* thing that would change is that replacing an item by assignment would only affect the *location* assigned to -- all other locations would not be affected. That really is what people *generally* want. If the entire list is meant to be read only -- the change would affect *nothing* at all. See if you can find *any* python program where people desired the multiplication to have the die effect that changing an object in one of the sub lists -- changes all the objects in the other sub lists. I'm sure you're not going to find it -- and even if you do, it's going to be 1 program in 1000's. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Mon, 05 Nov 2012 21:51:24 -0800, Andrew Robinson wrote: The most compact notation in programming really ought to reflect the most *commonly* desired operation. Otherwise, we're really just making people do extra typing for no reason. There are many reasons not to put minimizing of typing ahead of all other values: * Typically, code is written once and read many times. Minimizing typing might save you a second or two once, and then cost you many seconds every time you read the code. That's why we tell people to choose meaningful variable names, instead of naming everything a and b. * Consistency of semantics is better than a plethora of special cases. Python has a very simple and useful rule: objects should not be copied unless explicitly requested to be copied. This is much better than having to remember whether this operation or that operation makes a copy. The answer is consistent: (pardon me for belabouring the point here) Q: Does [0]*10 make ten copies of the integer object? A: No, list multiplication doesn't make copies of elements. Q: How about [0.0]*10? A: No, the elements are never copied. Q: What if I use a singleton? Does [None]*10 try to copy? A: No, the elements are never copied. Q: How about things like file objects that can't be copied? A: No, the elements are never copied. Q: What about [[]]*10? A: No, the elements are never copied. Q: How about if the elements are subclasses of list? A: No, the elements are never copied. Q: What about other mutable objects like sets or dicts? A: No, the elements are never copied. Q: What about instances of custom classes? A: No, the elements are never copied. Q: What if they are old-style Classic classes? A: No, the elements are never copied. Q: What if I do some funny tricks with the metaclass? A: No, the elements are never copied. Q: How about on Tuesdays? I bet they're copied on Tuesdays. A: No, the elements are never copied. Your proposal throws away consistency for a trivial benefit on a rare use- case, and replaces it with a bunch of special cases: Q: What about [[]]*10? A: Oh yeah, I forgot about lists, they're copied. Q: How about if the elements are subclasses of list? A: Hmmm, that's a good one, I'm not actually sure. Q: How about if I use delegation to proxy a list? A: Oh no, they definitely won't be copied. Q: What about other mutable objects like sets or dicts? A: No, definitely not. Unless people complain enough. Q: What about instances of custom classes? A: That's a definite maybe. Q: How about on Tuesdays? I bet they're copied on Tuesdays. A: Only if you're in Belgium. Losing consistency in favour of saving a few characters for something as uncommon as list multiplication is a poor tradeoff. That's why this proposal has been rejected again and again and again every time it has been suggested. List multiplication [x]*n is conceptually equivalent to: newlist = [] for i in range(n): newlist.append(x) or if you prefer a list comp: [x for i in range(n)] This is nice and simple and efficient. Some objects cannot be copied at all. Copying other objects is slow and inefficient. Keeping list multiplication consistent, and fast, is MUCH more important than making it work as expected for the rare case of 2D arrays: [[0]*n]*m where there are other alternatives. Further, list comprehensions take quite a bit longer to run than low level copies; by a factor of roughly 10. SO, it really would be worth implementing the underlying logic -- even if it wasn't super easy. Copying those elements does not come for free. It is true that list multiplication can be much faster than a list comp. But that's because the list multiply doesn't have to inspect the elements, copy them, or engage the iteration machinery. Avoiding copying gives you a big saving: [steve@ando ~]$ python3.3 -m timeit -s x = range(1000) [x for _ in range(100)] # not copied 10 loops, best of 3: 11.9 usec per loop [steve@ando utilities]$ python3.3 -m timeit -s x = range(1000) [x[:] for _ in range(100)] # copied 1 loops, best of 3: 103 usec per loop So there's a factor of ten difference right there. If list multiplication had to make copies, it would lose much of its speed advantage. For large enough lists, or complicated enough objects, it would become slower than a list comprehension. It would be even slower if list multiplication had to inspect each element first and decide whether or not to copy. I really don't think doing a shallow copy of lists would break anyone's program. Anyone who is currently using list multiplication with mutable objects is expecting that they will be the same object, and relying on that fact. Otherwise they wouldn't be using list multiplication. You're suggesting a semantic change. Therefore they will be expecting
Re: Multi-dimensional list initialization
On Tue, Nov 6, 2012 at 1:21 AM, Andrew Robinson andr...@r3dsolutions.com wrote: If you nest it another time; [[[None]]]*4, the same would happen; all lists would be independent -- but the objects which aren't lists would be refrenced-- not copied. a=[[[alpha,beta]]]*4 would yield: a=[[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']]] and a[0][0]=1 would give [[1],[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta' rather than a=[[1], [1], [1], [1]] Or at another level down: a[0][0][0]=1 would give: a=[[[1, 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']] ] rather than a=[[[1, 'beta']], [[1, 'beta']], [[1, 'beta']], [[1, 'beta']]] You wrote shallow copy. When the outer-level list is multiplied, the mid-level lists would be copied. Because the copies are shallow, although the mid-level lists are copied, their contents are not. Thus the inner-level lists would still be all referencing the same list. To demonstrate: from copy import copy class ShallowCopyList(list): ... def __mul__(self, number): ... new_list = ShallowCopyList() ... for _ in range(number): ... new_list.extend(map(copy, self)) ... return new_list ... a = ShallowCopyList([[[alpha, beta]]]) a [[['alpha', 'beta']]] b = a * 4 b [[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']]] b[0][0][0] = 1 b [[[1, 'beta']], [[1, 'beta']], [[1, 'beta']], [[1, 'beta']]] b[0][0] = 1 b [[1], [[1, 'beta']], [[1, 'beta']], [[1, 'beta']]] This shows that assignments at the middle level are independent with a shallow copy on multiplication, but assignments at the inner level are not. In order to achieve the behavior you describe, a deep copy would be needed. That really is what people *generally* want. If the entire list is meant to be read only -- the change would affect *nothing* at all. The time and memory cost of the multiplication operation would become quadratic instead of linear. See if you can find *any* python program where people desired the multiplication to have the die effect that changing an object in one of the sub lists -- changes all the objects in the other sub lists. I'm sure you're not going to find it -- and even if you do, it's going to be 1 program in 1000's. Per the last thread where we discussed extremely rare scenarios, shouldn't you be rounding 1 in 1000s up to 20%? ;-) -- http://mail.python.org/mailman/listinfo/python-list
RE: Multi-dimensional list initialization
Well said Steve, I agree with you... -Shambhu -Original Message- From: Steven D'Aprano [mailto:steve+comp.lang.pyt...@pearwood.info] Sent: Tuesday, November 06, 2012 2:35 PM To: python-list@python.org Subject: Re: Multi-dimensional list initialization On Mon, 05 Nov 2012 21:51:24 -0800, Andrew Robinson wrote: The most compact notation in programming really ought to reflect the most *commonly* desired operation. Otherwise, we're really just making people do extra typing for no reason. There are many reasons not to put minimizing of typing ahead of all other values: * Typically, code is written once and read many times. Minimizing typing might save you a second or two once, and then cost you many seconds every time you read the code. That's why we tell people to choose meaningful variable names, instead of naming everything a and b. * Consistency of semantics is better than a plethora of special cases. Python has a very simple and useful rule: objects should not be copied unless explicitly requested to be copied. This is much better than having to remember whether this operation or that operation makes a copy. The answer is consistent: (pardon me for belabouring the point here) Q: Does [0]*10 make ten copies of the integer object? A: No, list multiplication doesn't make copies of elements. Q: How about [0.0]*10? A: No, the elements are never copied. Q: What if I use a singleton? Does [None]*10 try to copy? A: No, the elements are never copied. Q: How about things like file objects that can't be copied? A: No, the elements are never copied. Q: What about [[]]*10? A: No, the elements are never copied. Q: How about if the elements are subclasses of list? A: No, the elements are never copied. Q: What about other mutable objects like sets or dicts? A: No, the elements are never copied. Q: What about instances of custom classes? A: No, the elements are never copied. Q: What if they are old-style Classic classes? A: No, the elements are never copied. Q: What if I do some funny tricks with the metaclass? A: No, the elements are never copied. Q: How about on Tuesdays? I bet they're copied on Tuesdays. A: No, the elements are never copied. Your proposal throws away consistency for a trivial benefit on a rare use- case, and replaces it with a bunch of special cases: Q: What about [[]]*10? A: Oh yeah, I forgot about lists, they're copied. Q: How about if the elements are subclasses of list? A: Hmmm, that's a good one, I'm not actually sure. Q: How about if I use delegation to proxy a list? A: Oh no, they definitely won't be copied. Q: What about other mutable objects like sets or dicts? A: No, definitely not. Unless people complain enough. Q: What about instances of custom classes? A: That's a definite maybe. Q: How about on Tuesdays? I bet they're copied on Tuesdays. A: Only if you're in Belgium. Losing consistency in favour of saving a few characters for something as uncommon as list multiplication is a poor tradeoff. That's why this proposal has been rejected again and again and again every time it has been suggested. List multiplication [x]*n is conceptually equivalent to: newlist = [] for i in range(n): newlist.append(x) or if you prefer a list comp: [x for i in range(n)] This is nice and simple and efficient. Some objects cannot be copied at all. Copying other objects is slow and inefficient. Keeping list multiplication consistent, and fast, is MUCH more important than making it work as expected for the rare case of 2D arrays: [[0]*n]*m where there are other alternatives. Further, list comprehensions take quite a bit longer to run than low level copies; by a factor of roughly 10. SO, it really would be worth implementing the underlying logic -- even if it wasn't super easy. Copying those elements does not come for free. It is true that list multiplication can be much faster than a list comp. But that's because the list multiply doesn't have to inspect the elements, copy them, or engage the iteration machinery. Avoiding copying gives you a big saving: [steve@ando ~]$ python3.3 -m timeit -s x = range(1000) [x for _ in range(100)] # not copied 10 loops, best of 3: 11.9 usec per loop [steve@ando utilities]$ python3.3 -m timeit -s x = range(1000) [x[:] for _ in range(100)] # copied 1 loops, best of 3: 103 usec per loop So there's a factor of ten difference right there. If list multiplication had to make copies, it would lose much of its speed advantage. For large enough lists, or complicated enough objects, it would become slower than a list comprehension. It would be even slower if list multiplication had to inspect each element first and decide whether or not to copy. I really don't think doing a shallow copy of lists would break anyone's program. Anyone who
Re: Multi-dimensional list initialization
On Nov 6, 2012 6:00 AM, Andrew Robinson andr...@r3dsolutions.com wrote: On 11/05/2012 06:30 PM, Oscar Benjamin wrote: stuff = [[obj] * n] * m I thought that the multiplication of the inner list ([obj] * n) by m could create a new list of lists using copies. On closer inspection I see that the list being multiplied is in fact [[obj] * n] and that this list can only know that it is a list of lists by inspecting its element(s) which makes things more complicated. I retract my claim that this change would be easy to implement. In general, people don't use element multiplication (that I have *ever* seen) to make lists where all elements of the outer most list point to the same sub-*list* by reference. The most common use of the multiplication is to fill an array with a constant, or short list of constants; Hence, almost everyone has to work around the issue as the initial poster did by using a much longer construction. That's what I have seen as well. I've never seen an example where someone wanted this behaviour. The most compact notation in programming really ought to reflect the most *commonly* desired operation. Otherwise, we're really just making people do extra typing for no reason. It's not so much the typing as the fact that this a common gotcha. Apparently many people expect different behaviour here. I seem to remember finding this surprising at first. Further, list comprehensions take quite a bit longer to run than low level copies; by a factor of roughly 10. SO, it really would be worth implementing the underlying logic -- even if it wasn't super easy. I really don't think doing a shallow copy of lists would break anyone's program. The non-list elements, whatever they are, can be left as reference copies -- but any element which is a list ought to be shallow copied. The behavior observed in the opening post where modifying one element of a sub-list, modifies all elements of all sub-lists is never desired as far as I have ever witnessed. It is a semantic change that would, I imagine, break many things in subtle ways. The underlying implementation of Python can check an object type trivially, and the only routine needed is a shallow list copy. So, no it really isn't a complicated operation to do shallow copies of lists. Yes but if you're inspecting the object to find out whether to copy it what do you test for? If you check for a list type what about subclasses? What if someone else has a custom list type that is not a subclass? Should there be a dunder method for this? I don't think it's such a simple problem. Oscar -- http://mail.python.org/mailman/listinfo/python-list
RE: Multi-dimensional list initialization
Ian Kelly wrote: On Tue, Nov 6, 2012 at 1:21 AM, Andrew Robinson [snip] See if you can find *any* python program where people desired the multiplication to have the die effect that changing an object in one of the sub lists -- changes all the objects in the other sub lists. I'm sure you're not going to find it -- and even if you do, it's going to be 1 program in 1000's. Per the last thread where we discussed extremely rare scenarios, shouldn't you be rounding 1 in 1000s up to 20%? ;-) Actually, I would be surprised if it was even 1 in 1000. Of course, consistency makes it easier to learn and *remember*. I value that far more than a minor quirk that is unlikely to bother me now that I know of it. Well, at least not as long as I do not forget my morning coffee/tea :) ~Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 06:35 AM, Oscar Benjamin wrote: In general, people don't use element multiplication (that I have *ever* seen) to make lists where all elements of the outer most list point to the same sub-*list* by reference. The most common use of the multiplication is to fill an array with a constant, or short list of constants; Hence, almost everyone has to work around the issue as the initial poster did by using a much longer construction. That's what I have seen as well. I've never seen an example where someone wanted this behaviour. The most compact notation in programming really ought to reflect the most *commonly* desired operation. Otherwise, we're really just making people do extra typing for no reason. It's not so much the typing as the fact that this a common gotcha. Apparently many people expect different behaviour here. I seem to remember finding this surprising at first. :) That's true as well. Further, list comprehensions take quite a bit longer to run than low level copies; by a factor of roughly 10. SO, it really would be worth implementing the underlying logic -- even if it wasn't super easy. I really don't think doing a shallow copy of lists would break anyone's program. The non-list elements, whatever they are, can be left as reference copies -- but any element which is a list ought to be shallow copied. The behavior observed in the opening post where modifying one element of a sub-list, modifies all elements of all sub-lists is never desired as far as I have ever witnessed. It is a semantic change that would, I imagine, break many things in subtle ways. ?? Do you have any guesses, how ? The underlying implementation of Python can check an object type trivially, and the only routine needed is a shallow list copy. So, no it really isn't a complicated operation to do shallow copies of lists. Yes but if you're inspecting the object to find out whether to copy it what do you test for? If you check for a list type what about subclasses? What if someone else has a custom list type that is not a subclass? Should there be a dunder method for this? No dunder methods. :) Custom non-subclass list types aren't a common usage for list multiplication in any event. At present one has to do list comprehensions for that, and that would simply remain so. Subclasses, however, are something I hadn't considered... I don't think it's such a simple problem. Oscar You made a good point, Oscar; I'll have to think about the subclassing a bit. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 09:32 AM, Prasad, Ramit wrote: Ian Kelly wrote: On Tue, Nov 6, 2012 at 1:21 AM, Andrew Robinson [snip] See if you can find *any* python program where people desired the multiplication to have the die effect that changing an object in one of the sub lists -- changes all the objects in the other sub lists. I'm sure you're not going to find it -- and even if you do, it's going to be 1 program in 1000's. Per the last thread where we discussed extremely rare scenarios, shouldn't you be rounding 1 in 1000s up to 20%? ;-) :D -- Ian -- also consider that I *am* willing to use extra memory. Not everything can be shrunk to nothing and still remain functional. :) So, it isn't *all* about *micro* optimization -- it's also about psychology and flexibility. Actually, I would be surprised if it was even 1 in 1000. Of course, consistency makes it easier to learn and *remember*. I value that far more than a minor quirk that is unlikely to bother me now that I know of it. Well, at least not as long as I do not forget my morning coffee/tea :) But, having it copy lists -- when the only purpose of multiplication is for lists; is only a minor quirk as well. ~Ramit -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 01:19 AM, Ian Kelly wrote: On Tue, Nov 6, 2012 at 1:21 AM, Andrew Robinson If you nest it another time; [[[None]]]*4, the same would happen; all lists would be independent -- but the objects which aren't lists would be refrenced-- not copied. a=[[[alpha,beta]]]*4 would yield: a=[[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']]] and a[0][0]=1 would give [[1],[['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta' rather than a=[[1], [1], [1], [1]] Or at another level down: a[0][0][0]=1 would give: a=[[[1, 'beta']], [['alpha', 'beta']], [['alpha', 'beta']], [['alpha', 'beta']] ] rather than a=[[[1, 'beta']], [[1, 'beta']], [[1, 'beta']], [[1, 'beta']]] You wrote shallow copy. When the outer-level list is multiplied, the mid-level lists would be copied. Because the copies are shallow, although the mid-level lists are copied, their contents are not. Thus the inner-level lists would still be all referencing the same list. To demonstrate: I meant all lists are shallow copied from the innermost level out. Equivalently, it's a deep copy of list objects -- but a shallow copy of any list contents except other lists. from copy import copy class ShallowCopyList(list): ... def __mul__(self, number): ... new_list = ShallowCopyList() ... for _ in range(number): ... new_list.extend(map(copy, self)) ... return new_list ... That type of copy is not equivalent to what I meant; It's a shallow copy only of non-list objects. This shows that assignments at the middle level are independent with a shallow copy on multiplication, but assignments at the inner level are not. In order to achieve the behavior you describe, a deep copy would be needed. Yes, it can be considered a deep copy of *all* list objects -- but not of non list contents. It's a terminology issue -- and you're right -- I need to be more precise. That really is what people *generally* want. If the entire list is meant to be read only -- the change would affect *nothing* at all. The time and memory cost of the multiplication operation would become quadratic instead of linear. Perhaps, but the copy would still not be _nearly_ as slow as a list comprehension !!! Being super fast when no one uses the output -- is , going nowhere fast. I think It's better to get at the right place at a medium speed than nowhere fast; List comprehensions *do* get to the right place, but *quite* slowly. They are both quadratic, *and* multiple tokenized steps. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Tue, Nov 6, 2012 at 2:36 PM, Andrew Robinson
andr...@r3dsolutions.com wrote:
I meant all lists are shallow copied from the innermost level out.
Equivalently, it's a deep copy of list objects -- but a shallow copy of any
list contents except other lists.
Why only list objects, though? When a user writes [[]] * 10, they
probably want a list containing ten distinct nested lists. Likewise,
when a user writes [{}] * 10, they probably want a list containing ten
distinct dicts, which is not at all an uncommon thing to want. It
seems very inconsistent that the former should work while the latter
should not. This is especially true when you start mixing the two
paradigms; the user might expect [[{}] * 10] * 10 to create a a 10x10
matrix where each element is a distinct dict, but this still would not
work, even though the nested lists would all have different
identities.
What about ([],) * 10? This is perhaps best interpreted as a request
to create a matrix of ten rows where the rows themselves are mutable
but the collection of rows is not. If list multiplication were to
copy nested lists, then should tuple multiplication do the same?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/06/2012 01:04 AM, Steven D'Aprano wrote:
On Mon, 05 Nov 2012 21:51:24 -0800, Andrew Robinson wrote:
The most compact notation in programming really ought to reflect the
most *commonly* desired operation. Otherwise, we're really just making
people do extra typing for no reason.
There are many reasons not to put minimizing of typing ahead of all other
values:
I didn't. I put it ahead of *some* values for the sake of practicality
and human psychology.
Practicality beats purity.
* Typically, code is written once and read many times. Minimizing
typing might save you a second or two once, and then cost you many
seconds every time you read the code. That's why we tell people to
choose meaningful variable names, instead of naming everything a
and b.
Yes. But this isn't going to cost any more time than figuring out
whether or not the list multiplication is going to cause quirks,
itself. Human psychology *tends* (it's a FAQ!) to automatically assume
the purpose of the list multiplication is to pre-allocate memory for the
equivalent (using lists) of a multi-dimensional array. Note the OP even
said 4d array.
The OP's original construction was simple, elegant, easy to read and
very commonly done by newbies learning the language because it's
*intuitive*. His second try was still intuitive, but less easy to read,
and not as elegant.
* Consistency of semantics is better than a plethora of special
cases. Python has a very simple and useful rule: objects should
not be copied unless explicitly requested to be copied. This is
much better than having to remember whether this operation or
that operation makes a copy. The answer is consistent:
Bull. Even in the last thread I noted the range() object produces
special cases.
range(0,5)[1]
1
range(0,5)[1:3]
range(1, 3)
The principle involved is that it gives you what you *usually* want; I
read some of the documentation on why Python 3 chose to implement it
this way.
(pardon me for belabouring the point here)
Q: Does [0]*10 make ten copies of the integer object?
A: No, list multiplication doesn't make copies of elements.
Neither would my idea for the vast majority of things on your first list.
Q: What about [[]]*10?
A: No, the elements are never copied.
YES! For the obvious reason that such a construction is making mutable
lists that the user wants to populate later. If they *didn't* want to
populate them later, they ought to have used tuples -- which take less
overhead. Who even does this thing you are suggesting?!
a=[[]]*10
a
[[], [], [], [], [], [], [], [], [], []]
a[0].append(1)
a
[[1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]
Oops! Damn, not what anyone normal wants
Q: How about if the elements are subclasses of list?
A: No, the elements are never copied.
Another poster brought that point up -- it's something I would have to
study before answering.
It's a valid objection.
Q: What about other mutable objects like sets or dicts?
A: No, the elements are never copied.
They aren't list multiplication compatible in any event! It's a total
nonsense objection.
If these are inconsistent in my idea -- OBVIOUSLY -- they are
inconsistent in Python's present implementation. You can't even
reference duplicate them NOW.
{ 1:'a', 2:'b', 3:'c' } * 2
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for *: 'dict' and 'int'
Q: How about on Tuesdays? I bet they're copied on Tuesdays.
A: No, the elements are never copied.
That's really a stupid objection, and everyone knows it.
Although that way may not be obvious at first unless you're Dutch.
Your proposal throws away consistency for a trivial benefit on a rare use-
case, and replaces it with a bunch of special cases:
RARE You are NUTS
Q: What about [[]]*10?
A: Oh yeah, I forgot about lists, they're copied.
Yup.
Q: How about if the elements are subclasses of list?
A: Hmmm, that's a good one, I'm not actually sure.
Q: How about if I use delegation to proxy a list?
A: Oh no, they definitely won't be copied.
Give an example usage of why someone would want to do this. Then we can
discuss it.
Q: What about other mutable objects like sets or dicts?
A: No, definitely not. Unless people complain enough.
now you're just repeating yourself to make your contrived list longer --
but there's no new objections...
Losing consistency in favour of saving a few characters for something as
uncommon as list multiplication is a poor tradeoff. That's why this
proposal has been rejected again and again and again every time it has
been suggested.
Please link to the objection being proposed to the developers, and their
reasoning for rejecting it.
I think you are exaggerating.
List multiplication [x]*n is conceptually equivalent to:
snip
This is nice and simple and efficient.
No
RE: Multi-dimensional list initialization
Andrew Robinson wrote:
On 11/06/2012 01:04 AM, Steven D'Aprano wrote:
On Mon, 05 Nov 2012 21:51:24 -0800, Andrew Robinson wrote:
[snip]
Q: What about other mutable objects like sets or dicts?
A: No, the elements are never copied.
They aren't list multiplication compatible in any event! It's a total
nonsense objection.
If these are inconsistent in my idea -- OBVIOUSLY -- they are
inconsistent in Python's present implementation. You can't even
reference duplicate them NOW.
{ 1:'a', 2:'b', 3:'c' } * 2
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for *: 'dict' and 'int'
z = [ {'a':1} ]*10
z[0]['b'] = 4
z
[{'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4},{'a': 1, 'b': 4},
{'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4}, {'a': 1, 'b': 4},
{'a': 1, 'b': 4}, {'a': 1, 'b': 4}]
Should that copy the dictionary? According to logical reasoning
it should copy the dictionary as well. How do you draw the line of
what should be copied and what should not?
Q: How about on Tuesdays? I bet they're copied on Tuesdays.
A: No, the elements are never copied.
That's really a stupid objection, and everyone knows it.
Agreed. [snip]
Q: How about if I use delegation to proxy a list?
A: Oh no, they definitely won't be copied.
Give an example usage of why someone would want to do this. Then we can
discuss it.
IIRC, someone wanted to do something very similar for dictionaries to
prevent editing of global variables.
Q: What about other mutable objects like sets or dicts?
A: No, definitely not. Unless people complain enough.
now you're just repeating yourself to make your contrived list longer --
but there's no new objections...
This is my main objection and one of the flaws of your argument.
You want to handle one type of mutable objects completely separately
than other mutable objects. Why is list any different than dictionary
in this respect? The only reason I can imagine is because lists
end up being used for 2d (or higher) matrices.
Losing consistency in favour of saving a few characters for something as
uncommon as list multiplication is a poor tradeoff. That's why this
proposal has been rejected again and again and again every time it has
been suggested.
Please link to the objection being proposed to the developers, and their
reasoning for rejecting it.
I think you are exaggerating.
I reject (as a developer) it because it forces me to remember a very
specific quirk versus a simple (logical) rule that applies to all objects. Not
to mention that the quirk is not even that useful except for beginners.
List multiplication [x]*n is conceptually equivalent to:
snip
This is nice and simple and efficient.
No it isn't efficient. It's *slow* when done as in your example.
Copying other objects is slow and inefficient. Keeping list
multiplication consistent, and fast, is MUCH more important than making
it work as expected for the rare case of 2D arrays:
I don't think so -- again, look at range(); it was made to work
inconsistent for a common case.
Besides, 2D arrays are *not* rare and people *have* to copy internals of
them very often.
The copy speed will be the same or *faster*, and the typing less -- and
the psychological mistakes *less*, the elegance more.
It's hardly going to confuse anyone to say that lists are copied with
list multiplication, but the elements are not.
Every time someone passes a list to a function, they *know* that the
list is passed by value -- and the elements are passed by reference.
People in Python are USED to lists being the way to weird behavior
that other languages don't do.
I think you just lost 90% of your credibility (with me). When did lists
get passed by value? Python uses call by sharing[0].
Terminology aside, lists are handled exactly the same way as all
other objects; the rules regarding their mutability in the callee
are the same as dictionaries, sets, or any mutable type (including
non-builtins).
Copying those elements does not come for free.
It is true that list multiplication can be much faster than a list comp.
But that's because the list multiply doesn't have to inspect the
elements, copy them, or engage the iteration machinery. Avoiding copying
gives you a big saving:
[steve@ando ~]$ python3.3 -m timeit -s x = range(1000)
[x for _ in range(100)] # not copied
10 loops, best of 3: 11.9 usec per loop
[steve@ando utilities]$ python3.3 -m timeit -s x = range(1000)
[x[:] for _ in range(100)] # copied
1 loops, best of 3: 103 usec per loop
So there's a factor of ten difference right there. If list multiplication
had to make copies, it would lose much of its speed advantage.
And when multiplication doesn't make copies of *lists*, it's going
nowhere fast, because people don't want the results that gives.
So what difference
Re: Multi-dimensional list initialization
On Tue, Nov 6, 2012 at 3:41 PM, Andrew Robinson
andr...@r3dsolutions.com wrote:
Q: What about other mutable objects like sets or dicts?
A: No, the elements are never copied.
They aren't list multiplication compatible in any event! It's a total
nonsense objection.
If these are inconsistent in my idea -- OBVIOUSLY -- they are inconsistent
in Python's present implementation. You can't even reference duplicate them
NOW.
{ 1:'a', 2:'b', 3:'c' } * 2
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for *: 'dict' and 'int'
The objection is not nonsense; you've merely misconstrued it. If
[[1,2,3]] * 4 is expected to create a mutable matrix of 1s, 2s, and
3s, then one would expect [[{}]] * 4 to create a mutable matrix of
dicts. If the dicts are not copied, then this fails for the same
reason
Q: How about if I use delegation to proxy a list?
A: Oh no, they definitely won't be copied.
Give an example usage of why someone would want to do this. Then we can
discuss it.
Seriously? Read a book on design patterns. You might start at SO:
http://stackoverflow.com/questions/832536/when-to-use-delegation-instead-of-inheritance
Losing consistency in favour of saving a few characters for something as
uncommon as list multiplication is a poor tradeoff. That's why this
proposal has been rejected again and again and again every time it has
been suggested.
Please link to the objection being proposed to the developers, and their
reasoning for rejecting it.
I think you are exaggerating.
From Google:
http://bugs.python.org/issue1408
http://bugs.python.org/issue12597
http://bugs.python.org/issue9108
http://bugs.python.org/issue7823
Note that in two out of these four cases, the reporter was trying to
multiply lists of dicts, not just lists of lists.
Besides, 2D arrays are *not* rare and people *have* to copy internals of
them very often.
The copy speed will be the same or *faster*, and the typing less -- and the
psychological mistakes *less*, the elegance more.
List multiplication is not potentially useful for copying 2D lists,
only for initializing them. For copying an existing nested list,
you're still stuck with either copy.deepcopy() or a list
comprehension.
It's hardly going to confuse anyone to say that lists are copied with list
multiplication, but the elements are not.
Every time someone passes a list to a function, they *know* that the list is
passed by value -- and the elements are passed by reference. People in
Python are USED to lists being the way to weird behavior that other
languages don't do.
Incorrect. Python uses what is commonly known as call-by-object, not
call-by-value or call-by-reference. Passing the list by value would
imply that the list is copied, and that appends or removes to the list
inside the function would not affect the original list. This is not
what Python does; the list inside the function and the list passed in
are the same list. At the same time, the function does not have
access to the original reference to the list and cannot reassign it by
reassigning its own reference, so it is not call-by-reference
semantics either.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 2012-11-06 23:52, Ian Kelly wrote:
On Tue, Nov 6, 2012 at 3:41 PM, Andrew Robinson
andr...@r3dsolutions.com wrote:
Q: What about other mutable objects like sets or dicts?
A: No, the elements are never copied.
They aren't list multiplication compatible in any event! It's a total
nonsense objection.
If these are inconsistent in my idea -- OBVIOUSLY -- they are inconsistent
in Python's present implementation. You can't even reference duplicate them
NOW.
{ 1:'a', 2:'b', 3:'c' } * 2
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: unsupported operand type(s) for *: 'dict' and 'int'
The objection is not nonsense; you've merely misconstrued it. If
[[1,2,3]] * 4 is expected to create a mutable matrix of 1s, 2s, and
3s, then one would expect [[{}]] * 4 to create a mutable matrix of
dicts. If the dicts are not copied, then this fails for the same
reason
Q: How about if I use delegation to proxy a list?
A: Oh no, they definitely won't be copied.
Give an example usage of why someone would want to do this. Then we can
discuss it.
Seriously? Read a book on design patterns. You might start at SO:
http://stackoverflow.com/questions/832536/when-to-use-delegation-instead-of-inheritance
Losing consistency in favour of saving a few characters for something as
uncommon as list multiplication is a poor tradeoff. That's why this
proposal has been rejected again and again and again every time it has
been suggested.
Please link to the objection being proposed to the developers, and their
reasoning for rejecting it.
I think you are exaggerating.
From Google:
http://bugs.python.org/issue1408
http://bugs.python.org/issue12597
http://bugs.python.org/issue9108
http://bugs.python.org/issue7823
Note that in two out of these four cases, the reporter was trying to
multiply lists of dicts, not just lists of lists.
Besides, 2D arrays are *not* rare and people *have* to copy internals of
them very often.
The copy speed will be the same or *faster*, and the typing less -- and the
psychological mistakes *less*, the elegance more.
List multiplication is not potentially useful for copying 2D lists,
only for initializing them. For copying an existing nested list,
you're still stuck with either copy.deepcopy() or a list
comprehension.
It's hardly going to confuse anyone to say that lists are copied with list
multiplication, but the elements are not.
Every time someone passes a list to a function, they *know* that the list is
passed by value -- and the elements are passed by reference. People in
Python are USED to lists being the way to weird behavior that other
languages don't do.
Incorrect. Python uses what is commonly known as call-by-object, not
call-by-value or call-by-reference. Passing the list by value would
imply that the list is copied, and that appends or removes to the list
inside the function would not affect the original list. This is not
what Python does; the list inside the function and the list passed in
are the same list. At the same time, the function does not have
access to the original reference to the list and cannot reassign it by
reassigning its own reference, so it is not call-by-reference
semantics either.
I prefer the term reference semantics.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Tue, 06 Nov 2012 14:41:24 -0800, Andrew Robinson wrote: Yes. But this isn't going to cost any more time than figuring out whether or not the list multiplication is going to cause quirks, itself. Human psychology *tends* (it's a FAQ!) to automatically assume the purpose of the list multiplication is to pre-allocate memory for the equivalent (using lists) of a multi-dimensional array. Note the OP even said 4d array. I'm not entirely sure what your point is here. The OP screwed up -- he didn't generate a 4-dimensional array. He generated a 2-dimensional array. If his intuition about the number of dimensions is so poor, why should his intuition about list multiplication be treated as sacrosanct? As they say, the only truly intuitive interface is the nipple. There are many places where people's intuition about programming fail. And many places where Fred's intuition is the opposite of Barney's intuition. Even more exciting, there are places where people's intuition is *inconsistent*, where they expect a line of code to behave differently depending on their intention, rather than on the code. And intuition is often sub-optimal: e.g. isn't it intuitively obvious that 42 + 1 should give 43? (Unless it is intuitively obvious that it should give 421.) So while I prefer intuitively obvious behaviour where possible, it is not the holy grail, and I am quite happy to give it up. The OP's original construction was simple, elegant, easy to read and very commonly done by newbies learning the language because it's *intuitive*. His second try was still intuitive, but less easy to read, and not as elegant. Yes. And list multiplication is one of those areas where intuition is suboptimal -- it produces a worse outcome overall, even if one minor use- case gets a better outcome. I'm not disputing that [[0]*n]*m is intuitively obvious and easy. I'm disputing that this matters. Python would be worse off if list multiplication behaved intuitively. An analogy: the intuitively obvious thing to do with a screw is to bang it in with a hammer. It's long, thin, has a point at the end, and a flat head that just screams hit me. But if you do the intuitive thing, your carpentry will be *much worse* than the alternatives -- a hammered in screw holds much less strongly than either a nail or a screwed in screw. The surface area available for gripping is about 2% compared to a nail and about 0.01% compared to a screw used correctly. Having list multiplication copy has consequences beyond 2D arrays. Those consequences make the intuitive behaviour you are requesting a negative rather than a positive. If that means that newbie programmers have to learn not to hammer screws in, so be it. It might be harder, slower, and less elegant to drill a pilot hole and then screw the screw in, but the overall result is better. * Consistency of semantics is better than a plethora of special cases. Python has a very simple and useful rule: objects should not be copied unless explicitly requested to be copied. This is much better than having to remember whether this operation or that operation makes a copy. The answer is consistent: Bull. Even in the last thread I noted the range() object produces special cases. range(0,5)[1] 1 range(0,5)[1:3] range(1, 3) What's the special case here? What do you think is copied? You take a slice of a tuple, you get a new tuple. You take a slice of a list, you get a new list. You take a slice of a range object, you get a new range object. I'm honestly not getting what you think is inconsistent about this. The principle involved is that it gives you what you *usually* want; Who is the you that decides what you usually want? And how do they know what is usual? Two-dimensional arrays in Python using lists are quite rare. Anyone who is doing serious numeric work where they need 2D arrays is using numpy, not lists. There are millions of people using Python, so it's hardly surprising that once or twice a year some newbie trips over this. But it's not something that people tend to trip over again and again and again, like C's assignment is an expression misfeature. I read some of the documentation on why Python 3 chose to implement it this way. What documentation is this? Because this is a design decision that goes all the way back to at least Python 1.5: [steve@ando ~]$ python1.5 Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam x = [[0]*5]*3 x[0][1] = 99 x [[0, 99, 0, 0, 0], [0, 99, 0, 0, 0], [0, 99, 0, 0, 0]] So I expect the design decision for Python 3 was we made the right decision before, there's no need to change it. (pardon me for belabouring the point here) Q: Does [0]*10 make ten copies of the integer object? A: No, list multiplication doesn't make copies of elements. Neither would my idea
Re: Multi-dimensional list initialization
On Nov 7, 5:26 am, MRAB pyt...@mrabarnett.plus.com wrote: I prefer the term reference semantics. Ha! That hits the nail on the head. To go back to the OP: On Nov 5, 11:28 am, Demian Brecht demianbre...@gmail.com wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] (Obviously, I could have just hardcoded the initialization, but I'm too lazy to type all that out ;)) The behaviour I encountered seems a little contradictory to me. [None] * 4 creates four distinct elements in a single array while [[None] * 4] * 4 creates one distinct array of four distinct elements, with three references to it: a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behaviour and if so, why? In my mind either result makes sense, but the inconsistency is what throws me off. m=[[None] * 2] * 3 is the same as m=[[None]*2, [None]*2, [None]*2] until one starts doing things like m[0][0] = 'm' So dont do it! And to get python to help you by saying the same that I am saying do m=((None) * 2) * 3 (well almost... its a bit more messy in practice) m=(((None,) * 2),)*3 After that try assigning to m[0][0] and python will kindly say NO! tl;dr version: reference semantics is ok assignment is ok (well up to a point) assignment + reference semantics is not -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Wed, 07 Nov 2012 00:23:44 +, MRAB wrote: Incorrect. Python uses what is commonly known as call-by-object, not call-by-value or call-by-reference. Passing the list by value would imply that the list is copied, and that appends or removes to the list inside the function would not affect the original list. This is not what Python does; the list inside the function and the list passed in are the same list. At the same time, the function does not have access to the original reference to the list and cannot reassign it by reassigning its own reference, so it is not call-by-reference semantics either. I prefer the term reference semantics. Oh good, because what the world needs is yet another name for the same behaviour. - call by sharing - call by object sharing - call by object reference - call by object - call by value, where values are references (according to the Java community) - call by reference, where references refer to objects, not variables (according to the Ruby community) - reference semantics Anything else? http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
In article 5099ec1d$0$21759$c3e8da3$76491...@news.astraweb.com, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Wed, 07 Nov 2012 00:23:44 +, MRAB wrote: Incorrect. Python uses what is commonly known as call-by-object, not call-by-value or call-by-reference. Passing the list by value would imply that the list is copied, and that appends or removes to the list inside the function would not affect the original list. This is not what Python does; the list inside the function and the list passed in are the same list. At the same time, the function does not have access to the original reference to the list and cannot reassign it by reassigning its own reference, so it is not call-by-reference semantics either. I prefer the term reference semantics. Oh good, because what the world needs is yet another name for the same behaviour. - call by sharing - call by object sharing - call by object reference - call by object - call by value, where values are references (according to the Java community) - call by reference, where references refer to objects, not variables (according to the Ruby community) - reference semantics Anything else? http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing Call by social network? The called function likes the object. Depending on how it feels, it can also comment on some of the object's attributes. -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Roy Smith wrote: Call by social network? The called function likes the object. Depending on how it feels, it can also comment on some of the object's attributes. And then finds that it has inadvertently shared all its private data with other functions accessing the object. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
If anything is to be done in this area, it would be better as an extension of list comprehensions, e.g. [[None times 5] times 10] which would be equivalent to [[None for _i in xrange(5)] for _j in xrange(10)] -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 2012-11-06, at 5:55 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: I'm not entirely sure what your point is here. The OP screwed up -- he didn't generate a 4-dimensional array. He generated a 2-dimensional array. If his intuition about the number of dimensions is so poor, why should his intuition about list multiplication be treated as sacrosanct? Yep, I may have mis-worded the explanation a bit (although I *did* express that it was a 4D matrix in the OP). I was using a 2D list to represent a 4D matrix in order to easily iterate over 90 degree rotations with zip(*matrix[::-1]). It wasn't for production code (otherwise I *would* be using numpy), it was for an online programming challenge in which external libs are not supported. As they say, the only truly intuitive interface is the nipple. There are many places where people's intuition about programming fail. And many places where Fred's intuition is the opposite of Barney's intuition. I couldn't agree more with this. My question was *not* based on what I perceive to be intuitive (although most of this thread has now seemed to devolve into that and become more of a philosophical debate), but was based on what I thought may have been inconsistent behaviour (which was quickly cleared up with None being immutable and causing it to *seem* that the behaviour was inconsistent to the forgetful mind). As you touch on here, intuition is entirely subjective. If you're coming from a C/C++ background, I'd think that your intuition would be that everything's passed by value unless explicitly stated. Someone coming from another background (Lua perhaps?) would likely have entirely different intuition. So while I prefer intuitively obvious behaviour where possible, it is not the holy grail, and I am quite happy to give it up. I fail to see where there has been any giving up on intuitiveness in the context of this particular topic. In my mind, intuitiveness is generally born of repetitiveness and consistency. As everything in Python is a reference, it would seem to me to be inconsistent to treat expressions such as [[obj]*4]*4 un-semantically (Pythonically speaking) and making it *less* intuitive. I agree that Python would definitely be worse off. Demian Brecht @demianbrecht http://demianbrecht.github.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Mon, Nov 5, 2012 at 6:54 PM, Andrew Robinson andr...@r3dsolutions.com wrote: On 11/04/2012 11:27 PM, Chris Angelico wrote: On Mon, Nov 5, 2012 at 6:07 PM, Chris Rebertc...@rebertia.com wrote: x = None x.a = 42 Traceback (most recent call last): File stdin, line 1, inmodule AttributeError: 'NoneType' object has no attribute 'a' Python needs a YouGottaBeKiddingMeError for times when you do something utterly insane like this. Attributes of None??!? :) ChrisA Hmmm? Everything in Python is an object. Therefore! SURE. None *does* have attributes! ( even if not useful ones... ) eg: None.__getattribute__( __doc__ ) doesn't produce an error. Eh, I meant mutating None's attributes, which is just as insane as I said. In C, in Linux, at the end of the file errno.h, where all error codes are listed eg:( EIO, EAGAIN, EBUSY, E) They had a final error like the one you dreamed up, it was called EIEIO; and the comment read something like, All the way around Elmer's barn. There's been a collection of those around the place. A few memorable ones: EMILYPOST: Bad fork() ETOBACCO: Read on empty pipe EHORSE: Mount failed I may be misremembering, but I'm sure the originals can be found at the other end of a web search. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 5/11/12 07:27:52, Demian Brecht wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] Or alternateively: m = [[None] * 4 for _ in range(4)] (Obviously, I could have just hardcoded the initialization, but I'm too lazy to type all that out ;)) The behaviour I encountered seems a little contradictory to me. [None] * 4 creates four distinct elements in a single array Actually, it creates a list with four references to the same object. But then, this object is immutable, so you won't notice that it's the same object. while [[None] * 4] * 4 creates one distinct array of four distinct elements, with three references to it: We usually phrase that as a list with four references to the same list. The first reference is not special in any way. a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behaviour Yes. and if so, why? In my mind either result makes sense, but the inconsistency is what throws me off. There's no inconsistency: in both cases you get a list with four references to the same object. The only difference is that in the fist case, the references are to an immutable object, so the fact that it's the same object won't hurt you. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
Le lundi 5 novembre 2012 07:28:00 UTC+1, Demian Brecht a écrit : So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] (Obviously, I could have just hardcoded the initialization, but I'm too lazy to type all that out ;)) The behaviour I encountered seems a little contradictory to me. [None] * 4 creates four distinct elements in a single array while [[None] * 4] * 4 creates one distinct array of four distinct elements, with three references to it: a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behaviour and if so, why? In my mind either result makes sense, but the inconsistency is what throws me off. Demian Brecht @demianbrecht http://demianbrecht.github.com -- You probably mean a two-dimensional matrix not a 4D matrix. def DefMatrix(nrow, ncol, val): ... return [[val] * ncol for i in range(nrow)] ... aa = DefMatrix(2, 3, 1.0) aa aa = DefMatrix(2, 3, 1.0) aa [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] aa[0][0] = 3.14 aa[1][2] = 2.718 aa [[3.14, 1.0, 1.0], [1.0, 1.0, 2.718]] bb = DefMatrix(2, 3, None) bb [[None, None, None], [None, None, None]] bb[0][0] = 3.14 bb[1][2] = 2.718 bb [[3.14, None, None], [None, None, 2.718]] jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 2012-11-04, at 10:44 PM, Andrew Robinson andr...@r3dsolutions.com wrote: but I think you meant: m = [[None] * 4, [None] * 4, [None] * 4, [None] *4 ] rather than: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] Yes, I meant the former, thanks for catching the typo. Demian Brecht @demianbrecht http://demianbrecht.github.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 2012-11-04, at 11:07 PM, Chris Rebert c...@rebertia.com wrote: However, unlike a list object (as in your latter example), the object `None` is completely immutable (and what's more, a singleton value), so you just-so-happen *not to be able to* run into the same problem of mutating an object (assignment to an index of a list constitutes mutation of that list) that is referenced in multiple places, for you cannot mutate None in the first place! Thanks for clearing that up Chris (and the link to the FAQ). I had thought about that after going to bed (D'oh.. None is immutable.. *That's* gotta be why). Demian Brecht @demianbrecht http://demianbrecht.github.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 5 November 2012 06:27, Demian Brecht demianbre...@gmail.com wrote: a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behaviour and if so, why? In my mind either result makes sense, but the inconsistency is what throws me off. z = [[None] * 4] goes to z = [x, x, x, x] where x = [y] where y = None AND THEN z[0] = 2 means z = [p, x, x, x] where p = 2 AND z[1][0] = 3 means x = [q] where q = 3 hence z = [2, [3], [3], [3]] -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 5 November 2012 09:13, Hans Mulder han...@xs4all.nl wrote: On 5/11/12 07:27:52, Demian Brecht wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] Or alternateively: m = [[None] * 4 for _ in range(4)] That's the way to do it. I've seen this question many times between here and the python-tutor list. It does seem to be a common gotcha. I was just thinking to myself that it would be a hard thing to change because the list would need to know how to instantiate copies of all the different types of the elements in the list. Then I realised it doesn't. It is simply a case of how the list multiplication operator is implemented and whether it chooses to use a reference to the same list or make a copy of that list. Since all of this is implemented within the same list type it is a relatively easy change to make (ignoring backward compatibility concerns). I don't see this non-copying list multiplication behaviour as contradictory but has anyone ever actually found a use for it? Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Tue, Nov 6, 2012 at 12:32 PM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: I was just thinking to myself that it would be a hard thing to change because the list would need to know how to instantiate copies of all the different types of the elements in the list. Then I realised it doesn't. It is simply a case of how the list multiplication operator is implemented and whether it chooses to use a reference to the same list or make a copy of that list. Since all of this is implemented within the same list type it is a relatively easy change to make (ignoring backward compatibility concerns). I don't see this non-copying list multiplication behaviour as contradictory but has anyone ever actually found a use for it? Stupid example of why it can't copy: bad = [open(test_file)] * 4 How do you clone something that isn't Plain Old Data? Ultimately, that's where the problem comes from. It's easy enough to clone something that's all scalars (strings, integers, None, etc) and non-recursive lists/dicts of scalars, but anything more complicated than that is rather harder. If you want a deep copy and are prepared to handle any issues that might result, you can do this: import copy a=[[2,3,4]] a.extend(copy.deepcopy(a)) a[0][1]=10 a [[2, 10, 4], [2, 3, 4]] And some things just won't work: bad.extend(copy.deepcopy(bad)) Traceback (most recent call last): File pyshell#17, line 1, in module bad.extend(copy.deepcopy(bad)) File C:\Python32\lib\copy.py, line 147, in deepcopy y = copier(x, memo) File C:\Python32\lib\copy.py, line 209, in _deepcopy_list y.append(deepcopy(a, memo)) File C:\Python32\lib\copy.py, line 166, in deepcopy rv = reductor(2) TypeError: cannot serialize '_io.TextIOWrapper' object The default behaviour is safe and reliable. When you want something other than the default, there are ways of doing it. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 6 November 2012 02:01, Chris Angelico ros...@gmail.com wrote: On Tue, Nov 6, 2012 at 12:32 PM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: I was just thinking to myself that it would be a hard thing to change because the list would need to know how to instantiate copies of all the different types of the elements in the list. Then I realised it doesn't. It is simply a case of how the list multiplication operator is implemented and whether it chooses to use a reference to the same list or make a copy of that list. Since all of this is implemented within the same list type it is a relatively easy change to make (ignoring backward compatibility concerns). I don't see this non-copying list multiplication behaviour as contradictory but has anyone ever actually found a use for it? Stupid example of why it can't copy: bad = [open(test_file)] * 4 How do you clone something that isn't Plain Old Data? Ultimately, that's where the problem comes from. It's easy enough to clone something that's all scalars (strings, integers, None, etc) and non-recursive lists/dicts of scalars, but anything more complicated than that is rather harder. That's not what I meant. But now you've made me realise that I was wrong about what I did mean. In the case of stuff = [[obj] * n] * m I thought that the multiplication of the inner list ([obj] * n) by m could create a new list of lists using copies. On closer inspection I see that the list being multiplied is in fact [[obj] * n] and that this list can only know that it is a list of lists by inspecting its element(s) which makes things more complicated. I retract my claim that this change would be easy to implement. Oscar -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/05/2012 06:30 PM, Oscar Benjamin wrote: On 6 November 2012 02:01, Chris Angelicoros...@gmail.com wrote: On Tue, Nov 6, 2012 at 12:32 PM, Oscar Benjamin oscar.j.benja...@gmail.com wrote: I was just thinking to myself that it would be a hard thing to change because the list would need to know how to instantiate copies of all the different types of the elements in the list. Then I realised it doesn't. It is simply a case of how the list multiplication operator is implemented and whether it chooses to use a reference to the same list or make a copy of that list. Since all of this is implemented within the same list type it is a relatively easy change to make (ignoring backward compatibility concerns). I don't see this non-copying list multiplication behaviour as contradictory but has anyone ever actually found a use for it? Stupid example of why it can't copy: bad = [open(test_file)] * 4 How do you clone something that isn't Plain Old Data? Ultimately, that's where the problem comes from. It's easy enough to clone something that's all scalars (strings, integers, None, etc) and non-recursive lists/dicts of scalars, but anything more complicated than that is rather harder. That's not what I meant. But now you've made me realise that I was wrong about what I did mean. In the case of stuff = [[obj] * n] * m I thought that the multiplication of the inner list ([obj] * n) by m could create a new list of lists using copies. On closer inspection I see that the list being multiplied is in fact [[obj] * n] and that this list can only know that it is a list of lists by inspecting its element(s) which makes things more complicated. I retract my claim that this change would be easy to implement. Oscar Hi Oscar, In general, people don't use element multiplication (that I have *ever* seen) to make lists where all elements of the outer most list point to the same sub-*list* by reference. The most common use of the multiplication is to fill an array with a constant, or short list of constants; Hence, almost everyone has to work around the issue as the initial poster did by using a much longer construction. The most compact notation in programming really ought to reflect the most *commonly* desired operation. Otherwise, we're really just making people do extra typing for no reason. Further, list comprehensions take quite a bit longer to run than low level copies; by a factor of roughly 10. SO, it really would be worth implementing the underlying logic -- even if it wasn't super easy. I really don't think doing a shallow copy of lists would break anyone's program. The non-list elements, whatever they are, can be left as reference copies -- but any element which is a list ought to be shallow copied. The behavior observed in the opening post where modifying one element of a sub-list, modifies all elements of all sub-lists is never desired as far as I have ever witnessed. The underlying implementation of Python can check an object type trivially, and the only routine needed is a shallow list copy. So, no it really isn't a complicated operation to do shallow copies of lists. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Tue, Nov 6, 2012 at 4:51 PM, Andrew Robinson andr...@r3dsolutions.com wrote: I really don't think doing a shallow copy of lists would break anyone's program. Well, it's a change, a semantic change. It's almost certainly going to break _something_. But for the sake of argument, we can suppose that the change could be made. Would it be the right thing to do? Shallow copying by default would result in extremely weird behaviour. All the same confusion would result, only instead of comparing [None]*4 with [[None]]*4, there'd be confusion over the difference between [[None]]*4 and [[[None]]]*4. I don't think it would help anything, and it'd result in a lot more work for no benefit. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/04/2012 10:27 PM, Demian Brecht wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] FYI: The behavior is the same in python 3.2 m=[[None]*4]*4 produces a nested list with all references being to the first instance of the inner list construction. I agree, the result is very counter-intuitive; hmmm... but I think you meant: m = [[None] * 4, [None] * 4, [None] * 4, [None] *4 ] rather than: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] ? :) ? I asked a why question on another thread, and watched several dodges to the main question; I'll be watching to see if you get anything other than That's the way it's defined in the API. IMHO -- that's not a real answer. My guess is that the original implementation never considered anything beyond a 1d list. :) A more precise related question might be: is there a way to force the replication operator to use copying rather than referencing? :/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Sun, Nov 4, 2012 at 10:27 PM, Demian Brecht demianbre...@gmail.com wrote: So, here I was thinking oh, this is a nice, easy way to initialize a 4D matrix (running 2.7.3, non-core libs not allowed): m = [[None] * 4] * 4 The way to get what I was after was: m = [[None] * 4, [None] * 4, [None] * 4, [None * 4]] (Obviously, I could have just hardcoded the initialization, but I'm too lazy to type all that out ;)) The behaviour I encountered seems a little contradictory to me. [None] * 4 creates four distinct elements in a single array while [[None] * 4] * 4 creates one distinct array of four distinct elements, with three references to it: Incorrect. In /both/ cases, the result is a list of length 4, whose elements are 4 (references to) the exact same object as the original list's element. Put simply, the list multiplication operator never copies objects; it just makes additional references to them. However, unlike a list object (as in your latter example), the object `None` is completely immutable (and what's more, a singleton value), so you just-so-happen *not to be able to* run into the same problem of mutating an object (assignment to an index of a list constitutes mutation of that list) that is referenced in multiple places, for you cannot mutate None in the first place!: x = None x.a = 42 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'NoneType' object has no attribute 'a' # it doesn't overload any mutating operators: type(None).__dict__.keys() ['__hash__', '__repr__', '__doc__'] # and it obviously has no instance variables, # so, we can't modify it in any way whatsoever! (Lists, on the other hand, define item assignment, .pop(), .remove(), and a few other mutator methods.) a = [None] * 4 a[0] = 'a' a ['a', None, None, None] m = [[None] * 4] * 4 m[0][0] = 'm' m [['m', None, None, None], ['m', None, None, None], ['m', None, None, None], ['m', None, None, None]] Is this expected behavior Yes. It's also a FAQ: http://docs.python.org/2/faq/programming.html#how-do-i-create-a-multidimensional-list and if so, why? It's a general (albeit AFAIK unstated) principle that Python never copies objects unless you explicitly ask it to. You have encountered one example of this rule in action. In my mind either result makes sense, but the inconsistency is what throws me off. It is perfectly consistent, once you understand what list multiplication actually does. Cheers, Chris -- http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On Mon, Nov 5, 2012 at 6:07 PM, Chris Rebert c...@rebertia.com wrote: x = None x.a = 42 Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'NoneType' object has no attribute 'a' Python needs a YouGottaBeKiddingMeError for times when you do something utterly insane like this. Attributes of None??!? :) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization
On 11/04/2012 11:27 PM, Chris Angelico wrote: On Mon, Nov 5, 2012 at 6:07 PM, Chris Rebertc...@rebertia.com wrote: x = None x.a = 42 Traceback (most recent call last): File stdin, line 1, inmodule AttributeError: 'NoneType' object has no attribute 'a' Python needs a YouGottaBeKiddingMeError for times when you do something utterly insane like this. Attributes of None??!? :) ChrisA Hmmm? Everything in Python is an object. Therefore! SURE. None *does* have attributes! ( even if not useful ones... ) eg: None.__getattribute__( __doc__ ) doesn't produce an error. In C, in Linux, at the end of the file errno.h, where all error codes are listed eg:( EIO, EAGAIN, EBUSY, E) They had a final error like the one you dreamed up, it was called EIEIO; and the comment read something like, All the way around Elmer's barn. :) The poster just hit that strange wall -- *all* built in types are injection proof; and that property is both good and bad... -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization trouble
[EMAIL PROTECTED] wrote: Hello I found this very strange; is it a bug, is it a feature, am I being naughty or what? foo = [[0, 0], [0, 0]] baz = [ [0]*2 ] * 2 ... Why on earth does foo and baz behave differently?? This is a frequently made mistake. try also: bumble = [[0]*2 for 0 in xrange(2)] Think hard about why that might be. Then try: [id(x) for x in foo] [id(x) for x in baz] [id(x) for x in bumble] Now check your upper scalp for lightbulbs. --Scott David Daniels [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization trouble
[EMAIL PROTECTED] wrote: Hello I found this very strange; is it a bug, is it a feature, am I being naughty or what? the repeat operator (*) creates a new list with references to the same inner objects, so you end up with a list containing multiple references to the same list. also see: http://pyfaq.infogami.com/how-do-i-create-a-multidimensional-list /F -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-dimensional list initialization trouble
An expression like this creates a list of integers: [0] * 2 [0, 0] But an expression like this creates list of references to the list named `foo': foo = [0, 0] baz = [foo] * 2 [foo, foo] So, setting baz[0][0] = 1, is really setting foo[0] = 1. There is only one instance of foo, but you have multiple references. Try a list comprehension to get the result you want: foo = [[0 for ii in range(2)] for jj in range(2)] -- http://mail.python.org/mailman/listinfo/python-list
