Re: To clarify how Python handles two equal objects
Jen Kris wrote: Avi, Your comments go farther afield than my original question, but you made some interesting additional points. For example, I sometimes work with the C API and sys.getrefcount may be helpful in deciding when to INCREF and DECREF. But that’s another issue. The situation I described in my original post is limited to a case such as x = y where both "x" and "y" are arrays – whether they are lists in Python, or from the array module – and the question in a compiled C extension is whether the assignment can be done simply by "x" taking the pointer to "y" rather than moving all the data from "y" into the memory buffer for "x" which, for a wide array, would be much more time consuming than just moving a pointer. The other advantage to doing it that way is if, as in my case, we perform a math operation on any element in "x" then Python expects that the same change to be reflected in "y." If I don’t use the same pointers then I would have to perform that operation twice – once for "x" and once for "y" – in addition to the expense of moving all the data. The answers I got from this post confirmed that it I can use the pointer if "y" is not re-defined to something else during the lifespan of "x." If it is then "x" has to be restored to its original pointer. I did it that way, and helpfully the compiler did not overrule me. I haven't done much with C extensions, but I don't think you'd need to do anything with "x" in that case. If something else is assigned to "y", "x" would still be a reference to the original object - why would it need to be "restored" to anything? Unless I've misunderstood what's going on here... -- Mark. -- https://mail.python.org/mailman/listinfo/python-list
RE: To clarify how Python handles two equal objects
<<< Frank Millman>>> My 'aha' moment came when I understood that a python object has only three properties - a type, an id, and a value. It does *not* have a name. Yes, Frank, it is a bit like how some people need to wrap their minds around a concept like an anonymous function. It has no name and for many purposes needs no name, but at the same time can be bound to one or more names if needed. Some people will encounter a situation where you need to pass a function to be called such as a key function for determining the sort order and they will create a new function first the usual way with a name and then pass it along by name. That works fine but the function then persists while not being used again and even choosing a name can have consequences if the same name is already in use and so on. Experienced programmers might use some kind of lambda expression in-line as the function as an object is then created, and passed along as a reference to be used and probably discarded once nothing refers to it. Similar things can happen if an object is created as part of a larger complex such as a list or deque of them. Each item can be referenced unambiguously without having a formal name. You can pop off the next one an use it or ask to take the fifth. It may acquire and lose names as the program runs but sometimes can have many names or none. Getting people to see that an object exists whether it has no name and to speak about them with that understanding can be a problem. We as humans seem to think we are our names. But I know my names have changed over the years partially because I moved between countries and I have several additional names used just for special purposes, and yet for some purposes I am simply a number. If I raise my class in a lecture hall where my name is not known, they may point at me or ask the guy in the blue short to talk. At the Department of Motor Vehicles, my name, like everyone else, is NEXT! The real point is what Python does, not what other situations require. Two objects are equal does not always mean what you think or want it to mean. What we are discussing here is two objects loosely of type "name" that are being compared not for whether THEY are themselves equal as in the same symbols composing them and perhaps living in the same namespace. The question was about two objects that contained references to another object or even to different parts of another object. As it happens, that can be a deeper question in which some parts are not a required aspect of the language and may be part of one of many possible implementations. And, as noted, the names can be detached from that reference too so a link is not permanent. Have we beaten this one to death yet? -Original Message- From: Python-list On Behalf Of Frank Millman Sent: Sunday, January 15, 2023 12:47 AM To: python-list@python.org Subject: Re: To clarify how Python handles two equal objects -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 2023-01-15 4:36 AM, Roel Schroeven wrote: Chris Angelico schreef op 15/01/2023 om 1:41: On Sun, 15 Jan 2023 at 11:38, Jen Kris wrote: > > Yes, in fact I asked my original question – "I discovered something about Python array handling that I would like to clarify" -- because I saw that Python did it that way. > Yep. This is not specific to arrays; it is true of all Python objects. Also, I suspect you're still thinking about things backwards, and am trying to lead you to a completely different way of thinking that actually does align with Python's object model. Indeen, I also still have the impression that Jen is thinking in terms of variables that are possible aliased such as you can have in a language like C, instead of objects with one or more names like we have in Python. Jens, in the Python model you really have to think of the objects largely independently of the names that are or are not referencing the objects. My 'aha' moment came when I understood that a python object has only three properties - a type, an id, and a value. It does *not* have a name. Frank Millman -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Chris Angelico schreef op 15/01/2023 om 1:41: On Sun, 15 Jan 2023 at 11:38, Jen Kris wrote: > > Yes, in fact I asked my original question – "I discovered something about Python array handling that I would like to clarify" -- because I saw that Python did it that way. > Yep. This is not specific to arrays; it is true of all Python objects. Also, I suspect you're still thinking about things backwards, and am trying to lead you to a completely different way of thinking that actually does align with Python's object model. Indeen, I also still have the impression that Jen is thinking in terms of variables that are possible aliased such as you can have in a language like C, instead of objects with one or more names like we have in Python. Jens, in the Python model you really have to think of the objects largely independently of the names that are or are not referencing the objects. -- "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Sun, 15 Jan 2023 at 11:38, Jen Kris wrote: > > Yes, in fact I asked my original question – "I discovered something about > Python array handling that I would like to clarify" -- because I saw that > Python did it that way. > Yep. This is not specific to arrays; it is true of all Python objects. Also, I suspect you're still thinking about things backwards, and am trying to lead you to a completely different way of thinking that actually does align with Python's object model. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Yes, in fact I asked my original question – "I discovered something about Python array handling that I would like to clarify" -- because I saw that Python did it that way. Jan 14, 2023, 15:51 by ros...@gmail.com: > On Sun, 15 Jan 2023 at 10:32, Jen Kris via Python-list > wrote: > >> The situation I described in my original post is limited to a case such as x >> = y ... the assignment can be done simply by "x" taking the pointer to "y" >> rather than moving all the data from "y" into the memory buffer for "x" >> > > It's not simply whether it *can* be done. It, in fact, *MUST* be done > that way. The ONLY meaning of "x = y" is that you now have a name "x" > which refers to whatever object is currently found under the name "y". > This is not an optimization, it is a fundamental of Python's object > model. This is true regardless of what kind of object this is; every > object must behave this way. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Sun, 15 Jan 2023 at 10:32, Jen Kris via Python-list wrote: > The situation I described in my original post is limited to a case such as x > = y ... the assignment can be done simply by "x" taking the pointer to "y" > rather than moving all the data from "y" into the memory buffer for "x" > It's not simply whether it *can* be done. It, in fact, *MUST* be done that way. The ONLY meaning of "x = y" is that you now have a name "x" which refers to whatever object is currently found under the name "y". This is not an optimization, it is a fundamental of Python's object model. This is true regardless of what kind of object this is; every object must behave this way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
RE: To clarify how Python handles two equal objects
Avi, Your comments go farther afield than my original question, but you made some interesting additional points. For example, I sometimes work with the C API and sys.getrefcount may be helpful in deciding when to INCREF and DECREF. But that’s another issue. The situation I described in my original post is limited to a case such as x = y where both "x" and "y" are arrays – whether they are lists in Python, or from the array module – and the question in a compiled C extension is whether the assignment can be done simply by "x" taking the pointer to "y" rather than moving all the data from "y" into the memory buffer for "x" which, for a wide array, would be much more time consuming than just moving a pointer. The other advantage to doing it that way is if, as in my case, we perform a math operation on any element in "x" then Python expects that the same change to be reflected in "y." If I don’t use the same pointers then I would have to perform that operation twice – once for "x" and once for "y" – in addition to the expense of moving all the data. The answers I got from this post confirmed that it I can use the pointer if "y" is not re-defined to something else during the lifespan of "x." If it is then "x" has to be restored to its original pointer. I did it that way, and helpfully the compiler did not overrule me. Jan 13, 2023, 18:41 by avi.e.gr...@gmail.com: > Jen, > > This may not be on target but I was wondering about your needs in this > category. Are all your data in a form where all in a cluster are the same > object type, such as floating point? > > Python has features designed to allow you to get multiple views on such > objects such as memoryview that can be used to say see an array as a matrix > of n rows by m columns, or m x n, or any other combo. And of course the > fuller numpy package has quite a few features. > > However, as you note, there is no guarantee that any reference to the data > may not shift away from it unless you build fairly convoluted logic or data > structures such as having an object that arranges to do something when you > try to remove it, such as tinkering with the __del__ method as well as > whatever method is used to try to set it to a new value. I guess that might > make sense for something like asynchronous programming including when setting > locks so multiple things cannot overlap when being done. > > Anyway, some of the packages like numpy are optimized in many ways but if you > want to pass a subset of sorts to make processing faster, I suspect you could > do things like pass a memoryview but it might not be faster than what you > build albeit probably more reliable and portable. > > I note another odd idea that others may have mentioned, with caution. > > If you load the sys module, you can CAREFULLY use code like this. > > a="Something Unique" > sys.getrefcount(a) > 2 > > Note if a==1 you will get some huge number of references and this is > meaningless. The 2 above is because asking about how many references also > references it. > > So save what ever number you have and see what happens when you make a second > reference or a third, and what happens if you delete or alter a reference: > > a="Something Unique" > sys.getrefcount(a) > 2 > b = a > sys.getrefcount(a) > 3 > sys.getrefcount(b) > 3 > c = b > d = a > sys.getrefcount(a) > 5 > sys.getrefcount(d) > 5 > del(a) > sys.getrefcount(d) > 4 > b = "something else" > sys.getrefcount(d) > 3 > > So, in theory, you could carefully write your code to CHECK the reference > count had not changed but there remain edge cases where a removed reference > is replaced by yet another new reference and you would have no idea. > > Avi > > > -Original Message- > From: Python-list On > Behalf Of Jen Kris via Python-list > Sent: Wednesday, January 11, 2023 1:29 PM > To: Roel Schroeven > Cc: python-list@python.org > Subject: Re: To clarify how Python handles two equal objects > > Thanks for your comments. After all, I asked for clarity so it’s not > pedantic to be precise, and you’re helping to clarify. > > Going back to my original post, > > mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] > arr1 = mx1[2] > > Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed > because while they are different names, they are the assigned same memory > location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both > names will be updated. > > That’s what I meant by "an operation on one is an operat
RE: To clarify how Python handles two equal objects
Axel and others, I can appreciate the comparison to a partially applied function but not in this case. Not that it matters, but this example is more like creating an object in something like machine learning and initializing parameters without adding data. Only when you ad data and call upon some transforms and so on, does it do something. This case is even more general. You create an object that does NOTHING. It simply holds a start/end/step set of up to three values. Lots of other functions will take this object as an argument. It can be used and reused any number of times. Strictly speaking, code like name[5:10:1] just creates a transient slice object and then uses that to get the answer. It is not delayed or partial as much as making one does nothing. Stefan mentioned functools.partial and that does create a bit of a curried function that wraps the data and holds on to it so invoking it sort of wakes the function up, with some or all data already accessible. A slice does not do that and needs some other functionality to use IT alongside whatever object you want to see a slice of. No special behavior was intended by me. I was illustrating how some methods of providing a selected view of an object are equally sensitive to the underlying data changing. A partially applied function that still takes an argument later, would have a similar problem if underlying data outside the what is stored within the function, changed, or if the saved was a reference to something that changed. But this is really far from unique. In the example given of creating a partial call, what if you made a second copy to that call then the first variable to the partial function was re-defined. -Original Message- From: Python-list On Behalf Of Axel Reichert Sent: Friday, January 13, 2023 3:22 AM To: python-list@python.org Subject: Re: To clarify how Python handles two equal objects writes: > As an example, you can create a named slice such as: > > middle_by_two = slice(5, 10, 2) > > The above is not in any sense pointing at anything yet. >From a functional programming point of view this just looks like a partially applied function, and with this in mind the behaviour to me seems to be completely as expected. No surprises here, or do I miss something? Best regards Axel -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: To clarify how Python handles two equal objects
Jen,
Can a compiler, or spot compiler, always know if something that was a reference
is changed?
Obvious examples may be if a change happens in non-deterministic ways such as
in a fork chosen at random or from user input but also sometimes levels of
indirection such as deleting an object that internally contains a reference the
other, perhaps even more indirectly.
I know programmers often have their code overhauled and their assumptions go
away such as someone deciding to create a variable name in an inner scope and
thus hiding the same variable they were using in an outer scope. So even if
your code is currently valid, after it changes, a later compiler if it detected
some change might not want to do your speedup.
I think the subject line of the message we keep exchanging is now a bit
misleading. It is not about two objects nor really about how python handles
them. There seem to be one object and possibly multiple views of it and you may
not want to pass the entire object around or manipulate it a certain way. I am
not so certain your methods necessarily speed things up as certain views simply
do calculations on the many places they need to read or change to supply what
you want.
From: Jen Kris
Sent: Friday, January 13, 2023 10:58 AM
To: avi.e.gr...@gmail.com
Cc: python-list@python.org
Subject: RE: To clarify how Python handles two equal objects
Avi,
Thanks for your comments. You make a good point.
Going back to my original question, and using your slice() example:
middle_by_two = slice(5, 10, 2)
nums = [n for n in range(12)]
q = nums[middle_by_two]
x = id(q)
b = q
y = id(b)
If I assign "b" to "q", then x and y match – they point to the same memory
until "b" OR "q" are reassigned to something else. If "q" changes during the
lifetime of "b" then it’s not safe to use the pointer to "q" for "b", as in:
nums = [n for n in range(2, 14)]
q = nums[middle_by_two]
x = id(q)
y = id(b)
Now "x" and "y" are different, as we would expect. So when writing a spot
speed up in a compiled language, you can see in the Python source if either is
reassigned, so you’ll know how to handle it. The motivation behind my question
was that in a compiled extension it’s faster to borrow a pointer than to move
an entire array if it’s possible, but special care must be taken.
Jen
Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com <mailto:avi.e.gr...@gmail.com> :
Jen,
It is dangerous territory you are treading as there are times all or parts of
objects are copied, or changed in place or the method you use to make a view is
not doing quite what you want.
As an example, you can create a named slice such as:
middle_by_two = slice(5, 10, 2)
The above is not in any sense pointing at anything yet. But given a long enough
list or other such objects, it will take items (starting at index 0) starting
with item that are at indices 5 then 7 then 9 as in this:
nums = [n for n in range(12)]
nums[middle_by_two]
[5, 7, 9]
The same slice will work on anything else:
list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
['f', 'h', 'j']
So although you may think the slice is bound to something, it is not. It is an
object that only later is briefly connected to whatever you want to apply it to.
If I later change nums, above, like this:
nums = [-3, -2, -1] + nums
nums
[-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
nums[middle_by_two]
[2, 4, 6]
In the example, you can forget about whether we are talking about pointers
directly or indirectly or variable names and so on. Your "view" remains valid
ONLY as long as you do not change either the slice or the underlying object you
are applying to -- at least not the items you want to extract.
Since my example inserted three new items at the start using negative numbers
for illustration, you would need to adjust the slice by making a new slice
designed to fit your new data. The example below created an adjusted slice that
adds 3 to the start and stop settings of the previous slice while copying the
step value and then it works on the elongated object:
middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 3,
middle_by_two.step)
nums[middle_by_two_adj]
[5, 7, 9]
A suggestion is that whenever you are not absolutely sure that the contents of
some data structure might change without your participation, then don't depend
on various kinds of aliases to keep the contents synchronized. Make a copy,
perhaps a deep copy and make sure the only thing ever changing it is your code
and later, if needed, copy the result back to any other data structure. Of
course, if anything else is accessing the result in the original in between, it
won't work.
Just FYI, a similar analysis applies to uses of the numpy an
RE: To clarify how Python handles two equal objects
Jen, This may not be on target but I was wondering about your needs in this category. Are all your data in a form where all in a cluster are the same object type, such as floating point? Python has features designed to allow you to get multiple views on such objects such as memoryview that can be used to say see an array as a matrix of n rows by m columns, or m x n, or any other combo. And of course the fuller numpy package has quite a few features. However, as you note, there is no guarantee that any reference to the data may not shift away from it unless you build fairly convoluted logic or data structures such as having an object that arranges to do something when you try to remove it, such as tinkering with the __del__ method as well as whatever method is used to try to set it to a new value. I guess that might make sense for something like asynchronous programming including when setting locks so multiple things cannot overlap when being done. Anyway, some of the packages like numpy are optimized in many ways but if you want to pass a subset of sorts to make processing faster, I suspect you could do things like pass a memoryview but it might not be faster than what you build albeit probably more reliable and portable. I note another odd idea that others may have mentioned, with caution. If you load the sys module, you can CAREFULLY use code like this. a="Something Unique" sys.getrefcount(a) 2 Note if a==1 you will get some huge number of references and this is meaningless. The 2 above is because asking about how many references also references it. So save what ever number you have and see what happens when you make a second reference or a third, and what happens if you delete or alter a reference: a="Something Unique" sys.getrefcount(a) 2 b = a sys.getrefcount(a) 3 sys.getrefcount(b) 3 c = b d = a sys.getrefcount(a) 5 sys.getrefcount(d) 5 del(a) sys.getrefcount(d) 4 b = "something else" sys.getrefcount(d) 3 So, in theory, you could carefully write your code to CHECK the reference count had not changed but there remain edge cases where a removed reference is replaced by yet another new reference and you would have no idea. Avi -Original Message- From: Python-list On Behalf Of Jen Kris via Python-list Sent: Wednesday, January 11, 2023 1:29 PM To: Roel Schroeven Cc: python-list@python.org Subject: Re: To clarify how Python handles two equal objects Thanks for your comments. After all, I asked for clarity so it’s not pedantic to be precise, and you’re helping to clarify. Going back to my original post, mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed because while they are different names, they are the assigned same memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both names will be updated. That’s what I meant by "an operation on one is an operation on the other." To be more precise, an operation on one name will be reflected in the other name. The difference is in the names, not the pointers. Each name has the same pointer in my example, but operations can be done in Python using either name. Jan 11, 2023, 09:13 by r...@roelschroeven.net: > Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: > >> Yes, I did understand that. In your example, "a" and "b" are the same >> pointer, so an operation on one is an operation on the other (because >> they’re the same memory block). >> > > Sorry if you feel I'm being overly pedantic, but your explanation "an > operation on one is an operation on the other (because they’re the same > memory block)" still feels a bit misguided. "One" and "other" still make it > sound like there are two objects, and "an operation on one" and "an operation > on the other" make it sound like there are two operations. > Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or > convenience, sometimes we really need to be precise. I think this is a case > where we need to be precise. > > So, to be precise: there is only one object, with possible multiple names to > it. We can change the object, using one of the names. That is one and only > one operation on one and only one object. Since the different names refer to > the same object, that change will of course be visible through all of them. > Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, > ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop > variables, function arguments. > > The correct mental model is important here, and I do think you're on track or > very close to it, but the way you phrase things does give me that nagging > fe
Re: To clarify how Python handles two equal objects
On 2023-01-13 16:57:45 +0100, Jen Kris via Python-list wrote: > Thanks for your comments. You make a good point. > > Going back to my original question, and using your slice() example: > > middle_by_two = slice(5, 10, 2) > nums = [n for n in range(12)] > q = nums[middle_by_two] > x = id(q) > b = q > y = id(b) > > If I assign "b" to "q", You don't asssign b to q. You assign q to b. Assignment is not commutative, the direction matters. > then x and y match – they point to the same memory until "b" OR "q" > are reassigned to something else. Correct. > If "q" changes during the lifetime of "b" then it’s not safe to use >the pointer to "q" for "b", as in: There is no pointer to q[1]. q is a pointer to something (an object of type list with 3 elements). b is a pointer to the same object at this point. Of course, if you assign a different pointer to q, then q will point to the new object from that point on while b will continue to point to the original object (until it is also re-assigned). > nums = [n for n in range(2, 14)] > q = nums[middle_by_two] > x = id(q) > y = id(b) > > Now "x" and "y" are different, as we would expect. > So when writing a spot speed up in a compiled language, If you are writing a Python compiler you certainly will have to adjust for the fact that the same variable may hold pointers to very different objects over its lifetime. But as far as I understand it, you aren't trying to write a Python compiler, just an extension, so that shouldn't concern you. > The motivation behind my question was that in a compiled extension > it’s faster to borrow a pointer than to move an entire array if it’s > possible, but special care must be taken. I still don't understand what you mean by that. hp [1] Not quite true. The run-time system must keep track of the variables, so there likely is a pointer to q somewhere. But it's (IMHO) irrelevant to this discussion, unless the extension is trying to look up variables by name or something like that. -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
; A suggestion is that whenever you are not absolutely sure that the >> contents of some data structure might change without your participation, >> then don't depend on various kinds of aliases to keep the contents >> synchronized. Make a copy, perhaps a deep copy and make sure the only thing >> ever changing it is your code and later, if needed, copy the result back to >> any other data structure. Of course, if anything else is accessing the >> result in the original in between, it won't work. >> > >> > Just FYI, a similar analysis applies to uses of the numpy and pandas and >> other modules if you get some kind of object holding indices to a series >> such as integers or Booleans and then later try using it after the number of >> items or rows or columns have changed. Your indices no longer match. >> > >> > Avi >> > >> > -Original Message- >> > From: Python-list > >> gmail@python.org>> > On Behalf Of Jen Kris via Python-list >> > Sent: Wednesday, January 11, 2023 1:29 PM >> > To: Roel Schroeven <>> r...@roelschroeven.net>> > >> > Cc: >> python-list@python.org >> > Subject: Re: To clarify how Python handles two equal objects >> > >> > Thanks for your comments. After all, I asked for clarity so it’s not >> pedantic to be precise, and you’re helping to clarify. >> > >> > Going back to my original post, >> > >> > mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] >> > arr1 = mx1[2] >> > >> > Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be >> changed because while they are different names, they are the assigned same >> memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then >> again both names will be updated. >> > >> > That’s what I meant by "an operation on one is an operation on the >> other." To be more precise, an operation on one name will be reflected in >> the other name. The difference is in the names, not the pointers. Each >> name has the same pointer in my example, but operations can be done in >> Python using either name. >> > >> > >> > >> > >> > Jan 11, 2023, 09:13 by >> r...@roelschroeven.net>> : >> > >> >> Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: >> >> >> >>> Yes, I did understand that. In your example, "a" and "b" are the same >> pointer, so an operation on one is an operation on the other (because >> they’re the same memory block). >> >>> >> >> >> >> Sorry if you feel I'm being overly pedantic, but your explanation "an >> operation on one is an operation on the other (because they’re the same >> memory block)" still feels a bit misguided. "One" and "other" still make it >> sound like there are two objects, and "an operation on one" and "an >> operation on the other" make it sound like there are two operations. >> >> Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity >> or convenience, sometimes we really need to be precise. I think this is a >> case where we need to be precise. >> >> >> >> So, to be precise: there is only one object, with possible multiple >> names to it. We can change the object, using one of the names. That is one >> and only one operation on one and only one object. Since the different names >> refer to the same object, that change will of course be visible through all >> of them. >> >> Note that 'name' in that sentence doesn't just refer to variables (mx1, >> arr1, ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), >> loop variables, function arguments. >> >> >> >> The correct mental model is important here, and I do think you're on >> track or very close to it, but the way you phrase things does give me that >> nagging feeling that you still might be just a bit off. >> >> >> >> -- >> >> "Peace cannot be kept by force. It can only be achieved through >> understanding." >> >> -- Albert Einstein >> >> >> >> -- >> >> >> https://mail.python.org/mailman/listinfo/python-list >> >> >> > >> > -- >> > >> https://mail.python.org/mailman/listinfo/python-list >> > >> >> -- >> >> https://mail.python.org/mailman/listinfo/python-list >> > > > -- > > Listen to my FREE CD at > http://www.mellowood.ca/music/cedars> > Bob van der Poel ** Wynndel, British Columbia, CANADA ** > EMAIL: > b...@mellowood.ca > WWW: > http://www.mellowood.ca > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
writes: > As an example, you can create a named slice such as: > > middle_by_two = slice(5, 10, 2) > > The above is not in any sense pointing at anything yet. >From a functional programming point of view this just looks like a partially applied function, and with this in mind the behaviour to me seems to be completely as expected. No surprises here, or do I miss something? Best regards Axel -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
It seems to me that the the entire concept of relying on python's idea of
where an object is stored is just plain dangerous. A most simple example
might be:
>>> a=1
>>> b=1
>>> a is b
True
>>> a=1234
>>> b=1234
>>> a is b
False
Not sure what happens if you manipulate the data referenced by 'b' in the
first example thinking you are changing something referred to by 'a' ...
but you might be smart to NOT think that you know.
On Fri, Jan 13, 2023 at 9:00 AM Jen Kris via Python-list <
python-list@python.org> wrote:
>
> Avi,
>
> Thanks for your comments. You make a good point.
>
> Going back to my original question, and using your slice() example:
>
> middle_by_two = slice(5, 10, 2)
> nums = [n for n in range(12)]
> q = nums[middle_by_two]
> x = id(q)
> b = q
> y = id(b)
>
> If I assign "b" to "q", then x and y match – they point to the same memory
> until "b" OR "q" are reassigned to something else. If "q" changes during
> the lifetime of "b" then it’s not safe to use the pointer to "q" for "b",
> as in:
>
> nums = [n for n in range(2, 14)]
> q = nums[middle_by_two]
> x = id(q)
> y = id(b)
>
> Now "x" and "y" are different, as we would expect. So when writing a spot
> speed up in a compiled language, you can see in the Python source if either
> is reassigned, so you’ll know how to handle it. The motivation behind my
> question was that in a compiled extension it’s faster to borrow a pointer
> than to move an entire array if it’s possible, but special care must be
> taken.
>
> Jen
>
>
>
> Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com:
>
> > Jen,
> >
> > It is dangerous territory you are treading as there are times all or
> parts of objects are copied, or changed in place or the method you use to
> make a view is not doing quite what you want.
> >
> > As an example, you can create a named slice such as:
> >
> > middle_by_two = slice(5, 10, 2)
> >
> > The above is not in any sense pointing at anything yet. But given a long
> enough list or other such objects, it will take items (starting at index 0)
> starting with item that are at indices 5 then 7 then 9 as in this:
> >
> > nums = [n for n in range(12)]
> > nums[middle_by_two]
> >
> > [5, 7, 9]
> >
> > The same slice will work on anything else:
> >
> > list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
> > ['f', 'h', 'j']
> >
> > So although you may think the slice is bound to something, it is not. It
> is an object that only later is briefly connected to whatever you want to
> apply it to.
> >
> > If I later change nums, above, like this:
> >
> > nums = [-3, -2, -1] + nums
> > nums
> > [-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
> > nums[middle_by_two]
> > [2, 4, 6]
> >
> > In the example, you can forget about whether we are talking about
> pointers directly or indirectly or variable names and so on. Your "view"
> remains valid ONLY as long as you do not change either the slice or the
> underlying object you are applying to -- at least not the items you want to
> extract.
> >
> > Since my example inserted three new items at the start using negative
> numbers for illustration, you would need to adjust the slice by making a
> new slice designed to fit your new data. The example below created an
> adjusted slice that adds 3 to the start and stop settings of the previous
> slice while copying the step value and then it works on the elongated
> object:
> >
> > middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop +
> 3, middle_by_two.step)
> > nums[middle_by_two_adj]
> > [5, 7, 9]
> >
> > A suggestion is that whenever you are not absolutely sure that the
> contents of some data structure might change without your participation,
> then don't depend on various kinds of aliases to keep the contents
> synchronized. Make a copy, perhaps a deep copy and make sure the only
> thing ever changing it is your code and later, if needed, copy the result
> back to any other data structure. Of course, if anything else is accessing
> the result in the original in between, it won't work.
> >
> > Just FYI, a similar analysis applies to uses of the numpy and pandas and
> other modules if you get some kind of object holding indices to a series
> such as integers or Booleans and then later try using it after the number
> of items or rows or columns have changed. Your indices no longer match.
> >
> > Avi
> >
> &
RE: To clarify how Python handles two equal objects
Avi,
Thanks for your comments. You make a good point.
Going back to my original question, and using your slice() example:
middle_by_two = slice(5, 10, 2)
nums = [n for n in range(12)]
q = nums[middle_by_two]
x = id(q)
b = q
y = id(b)
If I assign "b" to "q", then x and y match – they point to the same memory
until "b" OR "q" are reassigned to something else. If "q" changes during the
lifetime of "b" then it’s not safe to use the pointer to "q" for "b", as in:
nums = [n for n in range(2, 14)]
q = nums[middle_by_two]
x = id(q)
y = id(b)
Now "x" and "y" are different, as we would expect. So when writing a spot
speed up in a compiled language, you can see in the Python source if either is
reassigned, so you’ll know how to handle it. The motivation behind my question
was that in a compiled extension it’s faster to borrow a pointer than to move
an entire array if it’s possible, but special care must be taken.
Jen
Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com:
> Jen,
>
> It is dangerous territory you are treading as there are times all or parts of
> objects are copied, or changed in place or the method you use to make a view
> is not doing quite what you want.
>
> As an example, you can create a named slice such as:
>
> middle_by_two = slice(5, 10, 2)
>
> The above is not in any sense pointing at anything yet. But given a long
> enough list or other such objects, it will take items (starting at index 0)
> starting with item that are at indices 5 then 7 then 9 as in this:
>
> nums = [n for n in range(12)]
> nums[middle_by_two]
>
> [5, 7, 9]
>
> The same slice will work on anything else:
>
> list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
> ['f', 'h', 'j']
>
> So although you may think the slice is bound to something, it is not. It is
> an object that only later is briefly connected to whatever you want to apply
> it to.
>
> If I later change nums, above, like this:
>
> nums = [-3, -2, -1] + nums
> nums
> [-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
> nums[middle_by_two]
> [2, 4, 6]
>
> In the example, you can forget about whether we are talking about pointers
> directly or indirectly or variable names and so on. Your "view" remains valid
> ONLY as long as you do not change either the slice or the underlying object
> you are applying to -- at least not the items you want to extract.
>
> Since my example inserted three new items at the start using negative numbers
> for illustration, you would need to adjust the slice by making a new slice
> designed to fit your new data. The example below created an adjusted slice
> that adds 3 to the start and stop settings of the previous slice while
> copying the step value and then it works on the elongated object:
>
> middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 3,
> middle_by_two.step)
> nums[middle_by_two_adj]
> [5, 7, 9]
>
> A suggestion is that whenever you are not absolutely sure that the contents
> of some data structure might change without your participation, then don't
> depend on various kinds of aliases to keep the contents synchronized. Make a
> copy, perhaps a deep copy and make sure the only thing ever changing it is
> your code and later, if needed, copy the result back to any other data
> structure. Of course, if anything else is accessing the result in the
> original in between, it won't work.
>
> Just FYI, a similar analysis applies to uses of the numpy and pandas and
> other modules if you get some kind of object holding indices to a series such
> as integers or Booleans and then later try using it after the number of items
> or rows or columns have changed. Your indices no longer match.
>
> Avi
>
> -Original Message-
> From: Python-list On
> Behalf Of Jen Kris via Python-list
> Sent: Wednesday, January 11, 2023 1:29 PM
> To: Roel Schroeven
> Cc: python-list@python.org
> Subject: Re: To clarify how Python handles two equal objects
>
> Thanks for your comments. After all, I asked for clarity so it’s not
> pedantic to be precise, and you’re helping to clarify.
>
> Going back to my original post,
>
> mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]
> arr1 = mx1[2]
>
> Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed
> because while they are different names, they are the assigned same memory
> location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both
> names will be updated.
>
> That’s what I meant by "an operation on one is an operation on the other."
> To be more precise, an operation on one name will
RE: To clarify how Python handles two equal objects
Jen,
It is dangerous territory you are treading as there are times all or parts of
objects are copied, or changed in place or the method you use to make a view is
not doing quite what you want.
As an example, you can create a named slice such as:
middle_by_two = slice(5, 10, 2)
The above is not in any sense pointing at anything yet. But given a long enough
list or other such objects, it will take items (starting at index 0) starting
with item that are at indices 5 then 7 then 9 as in this:
nums = [n for n in range(12)]
nums[middle_by_two]
[5, 7, 9]
The same slice will work on anything else:
list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
['f', 'h', 'j']
So although you may think the slice is bound to something, it is not. It is an
object that only later is briefly connected to whatever you want to apply it to.
If I later change nums, above, like this:
nums = [-3, -2, -1] + nums
nums
[-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
nums[middle_by_two]
[2, 4, 6]
In the example, you can forget about whether we are talking about pointers
directly or indirectly or variable names and so on. Your "view" remains valid
ONLY as long as you do not change either the slice or the underlying object you
are applying to -- at least not the items you want to extract.
Since my example inserted three new items at the start using negative numbers
for illustration, you would need to adjust the slice by making a new slice
designed to fit your new data. The example below created an adjusted slice that
adds 3 to the start and stop settings of the previous slice while copying the
step value and then it works on the elongated object:
middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 3,
middle_by_two.step)
nums[middle_by_two_adj]
[5, 7, 9]
A suggestion is that whenever you are not absolutely sure that the contents of
some data structure might change without your participation, then don't depend
on various kinds of aliases to keep the contents synchronized. Make a copy,
perhaps a deep copy and make sure the only thing ever changing it is your code
and later, if needed, copy the result back to any other data structure. Of
course, if anything else is accessing the result in the original in between, it
won't work.
Just FYI, a similar analysis applies to uses of the numpy and pandas and other
modules if you get some kind of object holding indices to a series such as
integers or Booleans and then later try using it after the number of items or
rows or columns have changed. Your indices no longer match.
Avi
-Original Message-
From: Python-list On
Behalf Of Jen Kris via Python-list
Sent: Wednesday, January 11, 2023 1:29 PM
To: Roel Schroeven
Cc: python-list@python.org
Subject: Re: To clarify how Python handles two equal objects
Thanks for your comments. After all, I asked for clarity so it’s not pedantic
to be precise, and you’re helping to clarify.
Going back to my original post,
mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]
arr1 = mx1[2]
Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed
because while they are different names, they are the assigned same memory
location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both
names will be updated.
That’s what I meant by "an operation on one is an operation on the other." To
be more precise, an operation on one name will be reflected in the other name.
The difference is in the names, not the pointers. Each name has the same
pointer in my example, but operations can be done in Python using either name.
Jan 11, 2023, 09:13 by r...@roelschroeven.net:
> Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list:
>
>> Yes, I did understand that. In your example, "a" and "b" are the same
>> pointer, so an operation on one is an operation on the other (because
>> they’re the same memory block).
>>
>
> Sorry if you feel I'm being overly pedantic, but your explanation "an
> operation on one is an operation on the other (because they’re the same
> memory block)" still feels a bit misguided. "One" and "other" still make it
> sound like there are two objects, and "an operation on one" and "an operation
> on the other" make it sound like there are two operations.
> Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or
> convenience, sometimes we really need to be precise. I think this is a case
> where we need to be precise.
>
> So, to be precise: there is only one object, with possible multiple names to
> it. We can change the object, using one of the names. That is one and only
> one operation on one and only one object. Since the different names refer to
> the same object, that change will of course be visible through all of t
Re: To clarify how Python handles two equal objects
On 2023-01-11 18:49:14 +, Stefan Ram wrote: > Jen Kris writes: > >Each name has the same pointer > > ... from the C programmer's point of view. > > From the Python programmer's point of view, there are no "pointers". That's just window dressing. Pointers are evil, so we can't have pointers. So we'll just call them by a different name. > Instead, names or other targets are /assigned/ or /bound/ to objects. This is the wrong way around. The name isn't assigned to the object. The object is assigned to the name. As you quoted: > |... the sequence is asked to assign the assigned object to > |its item with that index ... Also, while the word "bind" is used in this manner in the official docs > |... the name is bound to the object ... and some members of this list, I think it is really misleading. It sounds like the name is now an attribute of the object. But it isn't. There is no way to the name (or the names) from the object. But there is a way to get from the name the object. I can't think of a snappy, unambiguous single verb to describe what's happening here, but do we need one? We already have words like "assign", "refer", "point" with fairly standardized meaning in IT. We can write that "a reference to an object is assigned to a variable", and that after the assignment "the variable refers to the object". Or we can even use the P-word. And since we know that variables in Python can only contain references and never values, we can abbreviate "a reference to an object" as "an object" in most contexts. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Tue, 10 Jan 2023 16:59:59 -0500, Thomas Passin declaimed the following: >Just to add a possibly picky detail to what others have said, Python >does not have an "array" type. It has a "list" type, as well as some >other, not necessarily mutable, sequence types. > However, it has long had https://docs.python.org/3/library/array.html -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
I think the key point is that "the operation" doesn't act on "names" but on "objects" (which are different sort of things), and thus there isn't an "the other" when talking about the object being operated on. Thinking of an operation being on a "name" is the mental model error. The only operations that operate on "names" are assignment operations. Augmented assignment operations are more complicatd, as they can either work on the object the name points on, if that object is mutable, or rebinds the name to a new object if it isn't. Thus a += b is NOT neccessarilily the same as a = a + b, as a += b might just mutate the object that a is bound to or might rebind in the manner of a = a + b; Thus: a = [ 1, 2, 3] b = a a += [4] will change the single list that a and b are bound to into [1, 2, 3, 4], while a = "foo" b = a a += "bar" will change a to be bound to the string object "foobar" but not b, since the string object "foo" wasn't mutable. Brings up the point, that you need to be careful with augmented assignment operators. On 1/11/23 1:28 PM, Jen Kris via Python-list wrote: Thanks for your comments. After all, I asked for clarity so it’s not pedantic to be precise, and you’re helping to clarify. Going back to my original post, mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed because while they are different names, they are the assigned same memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both names will be updated. That’s what I meant by "an operation on one is an operation on the other." To be more precise, an operation on one name will be reflected in the other name. The difference is in the names, not the pointers. Each name has the same pointer in my example, but operations can be done in Python using either name. Jan 11, 2023, 09:13 by r...@roelschroeven.net: Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). Sorry if you feel I'm being overly pedantic, but your explanation "an operation on one is an operation on the other (because they’re the same memory block)" still feels a bit misguided. "One" and "other" still make it sound like there are two objects, and "an operation on one" and "an operation on the other" make it sound like there are two operations. Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or convenience, sometimes we really need to be precise. I think this is a case where we need to be precise. So, to be precise: there is only one object, with possible multiple names to it. We can change the object, using one of the names. That is one and only one operation on one and only one object. Since the different names refer to the same object, that change will of course be visible through all of them. Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop variables, function arguments. The correct mental model is important here, and I do think you're on track or very close to it, but the way you phrase things does give me that nagging feeling that you still might be just a bit off. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Thanks for your comments. After all, I asked for clarity so it’s not pedantic to be precise, and you’re helping to clarify. Going back to my original post, mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed because while they are different names, they are the assigned same memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both names will be updated. That’s what I meant by "an operation on one is an operation on the other." To be more precise, an operation on one name will be reflected in the other name. The difference is in the names, not the pointers. Each name has the same pointer in my example, but operations can be done in Python using either name. Jan 11, 2023, 09:13 by r...@roelschroeven.net: > Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: > >> Yes, I did understand that. In your example, "a" and "b" are the same >> pointer, so an operation on one is an operation on the other (because >> they’re the same memory block). >> > > Sorry if you feel I'm being overly pedantic, but your explanation "an > operation on one is an operation on the other (because they’re the same > memory block)" still feels a bit misguided. "One" and "other" still make it > sound like there are two objects, and "an operation on one" and "an operation > on the other" make it sound like there are two operations. > Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or > convenience, sometimes we really need to be precise. I think this is a case > where we need to be precise. > > So, to be precise: there is only one object, with possible multiple names to > it. We can change the object, using one of the names. That is one and only > one operation on one and only one object. Since the different names refer to > the same object, that change will of course be visible through all of them. > Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, > ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop > variables, function arguments. > > The correct mental model is important here, and I do think you're on track or > very close to it, but the way you phrase things does give me that nagging > feeling that you still might be just a bit off. > > -- > "Peace cannot be kept by force. It can only be achieved through > understanding." > -- Albert Einstein > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). Sorry if you feel I'm being overly pedantic, but your explanation "an operation on one is an operation on the other (because they’re the same memory block)" still feels a bit misguided. "One" and "other" still make it sound like there are two objects, and "an operation on one" and "an operation on the other" make it sound like there are two operations. Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or convenience, sometimes we really need to be precise. I think this is a case where we need to be precise. So, to be precise: there is only one object, with possible multiple names to it. We can change the object, using one of the names. That is one and only one operation on one and only one object. Since the different names refer to the same object, that change will of course be visible through all of them. Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop variables, function arguments. The correct mental model is important here, and I do think you're on track or very close to it, but the way you phrase things does give me that nagging feeling that you still might be just a bit off. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). My issue in Python came up because Python can dynamically change one or the other to a different object (memory block) so I have to be aware of that when handing this kind of situation. Jan 10, 2023, 17:31 by greg.ew...@canterbury.ac.nz: > On 11/01/23 11:21 am, Jen Kris wrote: > >> where one object derives from another object (a = b[0], for example), any >> operation that would alter one will alter the other. >> > > I think you're still confused. In C terms, after a = b[0], a and b[0] > are pointers to the same block of memory. If you change that block of > memory, then of course you will see the change through either pointer. > > Here's a rough C translation of some of your Python code: > > /* mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] */ > int **mx1 = (int **)malloc(3 * sizeof(int *)); > mx1[0] = (int *)malloc(3 * sizeof(int)); > mx1[0][0] = 1; > mx1[0][1] = 2; > mx1[0][2] = 3; > mx1[1] = (int *)malloc(3 * sizeof(int)); > mx1[1][0] = 4; > mx1[1][1] = 5; > mx1[1][2] = 6; > mx1[2] = (int *)malloc(3 * sizeof(int)); > mx1[2][0] = 7; > mx1[2][1] = 8; > mx1[2][2] = 9; > > /* arr1 = mx1[2] */ > int *arr1 = mx[2]; > > /* arr1 = [ 10, 11, 12 ] */ > arr1 = (int *)malloc(3 * sizeof(int)); > arr1[0] = 10; > arr1[1] = 11; > arr1[2] = 12; > > Does that help your understanding? > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 11/01/23 11:21 am, Jen Kris wrote: where one object derives from another object (a = b[0], for example), any operation that would alter one will alter the other. I think you're still confused. In C terms, after a = b[0], a and b[0] are pointers to the same block of memory. If you change that block of memory, then of course you will see the change through either pointer. Here's a rough C translation of some of your Python code: /* mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] */ int **mx1 = (int **)malloc(3 * sizeof(int *)); mx1[0] = (int *)malloc(3 * sizeof(int)); mx1[0][0] = 1; mx1[0][1] = 2; mx1[0][2] = 3; mx1[1] = (int *)malloc(3 * sizeof(int)); mx1[1][0] = 4; mx1[1][1] = 5; mx1[1][2] = 6; mx1[2] = (int *)malloc(3 * sizeof(int)); mx1[2][0] = 7; mx1[2][1] = 8; mx1[2][2] = 9; /* arr1 = mx1[2] */ int *arr1 = mx[2]; /* arr1 = [ 10, 11, 12 ] */ arr1 = (int *)malloc(3 * sizeof(int)); arr1[0] = 10; arr1[1] = 11; arr1[2] = 12; Does that help your understanding? -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 2023-01-10 22:21, Jen Kris via Python-list wrote: There are cases where NumPy would be the best choice, but that wasn’t the case here with what the loop was doing. To sum up what I learned from this post, where one object derives from another object (a = b[0], for example), any operation that would alter one will alter the other. When either is assigned to something else, then they no longer point to the same memory location and they’re once again independent. I hope the word "derives" sidesteps the semantic issue of whether they are "equal." [snip] In C terms (and in CPython), a 'list' is a resizable array of pointers to objects, so after "a=b[0]", the name "a" will point to the same object that b[0] points to. That object might or might not be mutable. -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 1/10/2023 5:21 PM, Jen Kris wrote: There are cases where NumPy would be the best choice, but that wasn’t the case here with what the loop was doing. To sum up what I learned from this post, where one object derives from another object (a = b[0], for example), any operation that would alter one will alter the other. Let's make sure we're clear here. The way you were doing it, it *looks like* "one alters the other". But in reality, both are the same thing, and so when that thing gets altered in some way, both variables will show the change because they are in fact references to the same object. As an analogy, if you dye your hair purple and look in a mirror, you will see your image with the new purple hair. If you look in a different mirror, you will also see your image with the new purple hair. They are both reflections of the same object, namely you with your new purple hair. The point about the identity of objects contained within other objects is echoed by the copy and deepcopy operations. copy() copies the references, deepcopy() makes new objects that are equal to the original ones. After making a deep copy of a list and assigning it to new variable, changes in one will no longer show up in the other because the elements are are no longer the same elements. When either is assigned to something else, then they no longer point to the same memory location and they’re once again independent. This is right except that in Python, it's better not to think about their memory locations, because that would basically be an implementation detail (well, except that if you are going to access them with C you will possibly need actual locations). Their logical identity would be better to think about. I hope the word "derives" sidesteps the semantic issue of whether they are "equal." Thanks to all who replied to this post. Jen Jan 10, 2023, 13:59 by li...@tompassin.net: Just to add a possibly picky detail to what others have said, Python does not have an "array" type. It has a "list" type, as well as some other, not necessarily mutable, sequence types. If you want to speed up list and matrix operations, you might use NumPy. Its arrays and matrices are heavily optimized for fast processing and provide many useful operations on them. No use calling out to C code yourself when NumPy has been refining that for many years. On 1/10/2023 4:10 PM, MRAB wrote: On 2023-01-10 20:41, Jen Kris via Python-list wrote: Thanks for your comments. I'd like to make one small point. You say: "Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way." But where they have been set to the same object, an operation on one will affect the other as long as they are equal (in Python). So I will have to conform them in those cases because Python will reflect any math operation in both the array and the matrix. It's not a 2D matrix, it's a 1D list containing references to 1D lists, each of which contains references to Python ints. In CPython, references happen to be pointers, but that's just an implementation detail. Jan 10, 2023, 12:28 by ros...@gmail.com: On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list wrote: I am writing a spot speedup in assembly language for a short but computation-intensive Python loop, and I discovered something about Python array handling that I would like to clarify. For a simplified example, I created a matrix mx1 and assigned the array arr1 to the third row of the matrix: mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] The pointers to these are now the same: ida = id(mx1[2]) - 140260325306880 idb = id(arr1) - 140260325306880 That’s great because when I encounter this in assembly or C, I can just borrow the pointer to row 3 for the array arr1, on the assumption that they will continue to point to the same object. Then when I do any math operations in arr1 it will be reflected in both arrays because they are now pointing to the same array: That's not an optimization; what you've done is set arr1 to be a reference to that object. But on the next iteration we assign arr1 to something else: arr1 = [ 10,
Re: To clarify how Python handles two equal objects
On 1/10/23 12:03, Jen Kris via Python-list wrote: > I am writing a spot speedup in assembly language for a short but computation-intensive Python > loop, and I discovered something about Python array handling that I would like to clarify. > But on the next iteration we assign arr1 to something else: > > arr1 = [ 10, 11, 12 ] > idc = id(arr1) – 140260325308160 > idd = id(mx1[2]) – 140260325306880 > > Now arr1 is no longer equal to mx1[2]... If you want to have `arr1` to still be `mx1[2]` (and consequently for `mx1[2]` to now be `[10, 11, 12]` you need to mutate `arr1` instead of reassigning it: arr1[:] = [10, 11, 12] -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 1/10/2023 5:11 PM, Chris Angelico wrote: On Wed, 11 Jan 2023 at 09:08, Thomas Passin wrote: Just to add a possibly picky detail to what others have said, Python does not have an "array" type. It has a "list" type, as well as some other, not necessarily mutable, sequence types. Just to be even pickier, Python DOES have an array type, but it's not the one the OP was using :) https://docs.python.org/3/library/array.html Ha! And here all this time I thought you got them from an add-on package. -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
There are cases where NumPy would be the best choice, but that wasn’t the case here with what the loop was doing. To sum up what I learned from this post, where one object derives from another object (a = b[0], for example), any operation that would alter one will alter the other. When either is assigned to something else, then they no longer point to the same memory location and they’re once again independent. I hope the word "derives" sidesteps the semantic issue of whether they are "equal." Thanks to all who replied to this post. Jen Jan 10, 2023, 13:59 by li...@tompassin.net: > Just to add a possibly picky detail to what others have said, Python does not > have an "array" type. It has a "list" type, as well as some other, not > necessarily mutable, sequence types. > > If you want to speed up list and matrix operations, you might use NumPy. Its > arrays and matrices are heavily optimized for fast processing and provide > many useful operations on them. No use calling out to C code yourself when > NumPy has been refining that for many years. > > On 1/10/2023 4:10 PM, MRAB wrote: > >> On 2023-01-10 20:41, Jen Kris via Python-list wrote: >> >>> >>> Thanks for your comments. I'd like to make one small point. You say: >>> >>> "Assignment in Python is a matter of object references. It's not >>> "conform them as long as they remain equal". You'll have to think in >>> terms of object references the entire way." >>> >>> But where they have been set to the same object, an operation on one will >>> affect the other as long as they are equal (in Python). So I will have to >>> conform them in those cases because Python will reflect any math operation >>> in both the array and the matrix. >>> >> It's not a 2D matrix, it's a 1D list containing references to 1D lists, each >> of which contains references to Python ints. >> >> In CPython, references happen to be pointers, but that's just an >> implementation detail. >> >>> >>> >>> Jan 10, 2023, 12:28 by ros...@gmail.com: >>> On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list wrote: > > I am writing a spot speedup in assembly language for a short but > computation-intensive Python loop, and I discovered something about > Python array handling that I would like to clarify. > > For a simplified example, I created a matrix mx1 and assigned the array > arr1 to the third row of the matrix: > > mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] > arr1 = mx1[2] > > The pointers to these are now the same: > > ida = id(mx1[2]) - 140260325306880 > idb = id(arr1) - 140260325306880 > > That’s great because when I encounter this in assembly or C, I can just > borrow the pointer to row 3 for the array arr1, on the assumption that > they will continue to point to the same object. Then when I do any math > operations in arr1 it will be reflected in both arrays because they are > now pointing to the same array: > That's not an optimization; what you've done is set arr1 to be a reference to that object. > But on the next iteration we assign arr1 to something else: > > arr1 = [ 10, 11, 12 ] > idc = id(arr1) – 140260325308160 > idd = id(mx1[2]) – 140260325306880 > > Now arr1 is no longer equal to mx1[2], and any subsequent operations in > arr1 will not affect mx1. > Yep, you have just set arr1 to be a completely different object. > So where I’m rewriting some Python code in a low level language, I can’t > assume that the two objects are equal because that equality will not > remain if either is reassigned. So if I do some operation on one array I > have to conform the two arrays for as long as they remain equal, I can’t > just do it in one operation because I can’t rely on the objects remaining > equal. > > Is my understanding of this correct? Is there anything I’m missing? > Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Wed, 11 Jan 2023 at 09:08, Thomas Passin wrote: > > Just to add a possibly picky detail to what others have said, Python > does not have an "array" type. It has a "list" type, as well as some > other, not necessarily mutable, sequence types. Just to be even pickier, Python DOES have an array type, but it's not the one the OP was using :) https://docs.python.org/3/library/array.html ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Just to add a possibly picky detail to what others have said, Python does not have an "array" type. It has a "list" type, as well as some other, not necessarily mutable, sequence types. If you want to speed up list and matrix operations, you might use NumPy. Its arrays and matrices are heavily optimized for fast processing and provide many useful operations on them. No use calling out to C code yourself when NumPy has been refining that for many years. On 1/10/2023 4:10 PM, MRAB wrote: On 2023-01-10 20:41, Jen Kris via Python-list wrote: Thanks for your comments. I'd like to make one small point. You say: "Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way." But where they have been set to the same object, an operation on one will affect the other as long as they are equal (in Python). So I will have to conform them in those cases because Python will reflect any math operation in both the array and the matrix. It's not a 2D matrix, it's a 1D list containing references to 1D lists, each of which contains references to Python ints. In CPython, references happen to be pointers, but that's just an implementation detail. Jan 10, 2023, 12:28 by ros...@gmail.com: On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list wrote: I am writing a spot speedup in assembly language for a short but computation-intensive Python loop, and I discovered something about Python array handling that I would like to clarify. For a simplified example, I created a matrix mx1 and assigned the array arr1 to the third row of the matrix: mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] The pointers to these are now the same: ida = id(mx1[2]) - 140260325306880 idb = id(arr1) - 140260325306880 That’s great because when I encounter this in assembly or C, I can just borrow the pointer to row 3 for the array arr1, on the assumption that they will continue to point to the same object. Then when I do any math operations in arr1 it will be reflected in both arrays because they are now pointing to the same array: That's not an optimization; what you've done is set arr1 to be a reference to that object. But on the next iteration we assign arr1 to something else: arr1 = [ 10, 11, 12 ] idc = id(arr1) – 140260325308160 idd = id(mx1[2]) – 140260325306880 Now arr1 is no longer equal to mx1[2], and any subsequent operations in arr1 will not affect mx1. Yep, you have just set arr1 to be a completely different object. So where I’m rewriting some Python code in a low level language, I can’t assume that the two objects are equal because that equality will not remain if either is reassigned. So if I do some operation on one array I have to conform the two arrays for as long as they remain equal, I can’t just do it in one operation because I can’t rely on the objects remaining equal. Is my understanding of this correct? Is there anything I’m missing? Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Jen Kris via Python-list schreef op 10/01/2023 om 21:41: But where they have been set to the same object, an operation on one will affect the other as long as they are equal (in Python). As long as they are *identical*, not equal. Identical as in having the same identity as Python defines it. I advise you to read Ned Batchelder's explanation about names and values in Python, or watch his presentation, to get a good understanding. See https://nedbatchelder.com/text/names1.html -- "Don't Panic." -- Douglas Adams, The Hitchhiker's Guide to the Galaxy -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 2023-01-10 20:41, Jen Kris via Python-list wrote: Thanks for your comments. I'd like to make one small point. You say: "Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way." But where they have been set to the same object, an operation on one will affect the other as long as they are equal (in Python). So I will have to conform them in those cases because Python will reflect any math operation in both the array and the matrix. It's not a 2D matrix, it's a 1D list containing references to 1D lists, each of which contains references to Python ints. In CPython, references happen to be pointers, but that's just an implementation detail. Jan 10, 2023, 12:28 by ros...@gmail.com: On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list wrote: I am writing a spot speedup in assembly language for a short but computation-intensive Python loop, and I discovered something about Python array handling that I would like to clarify. For a simplified example, I created a matrix mx1 and assigned the array arr1 to the third row of the matrix: mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] The pointers to these are now the same: ida = id(mx1[2]) - 140260325306880 idb = id(arr1) - 140260325306880 That’s great because when I encounter this in assembly or C, I can just borrow the pointer to row 3 for the array arr1, on the assumption that they will continue to point to the same object. Then when I do any math operations in arr1 it will be reflected in both arrays because they are now pointing to the same array: That's not an optimization; what you've done is set arr1 to be a reference to that object. But on the next iteration we assign arr1 to something else: arr1 = [ 10, 11, 12 ] idc = id(arr1) – 140260325308160 idd = id(mx1[2]) – 140260325306880 Now arr1 is no longer equal to mx1[2], and any subsequent operations in arr1 will not affect mx1. Yep, you have just set arr1 to be a completely different object. So where I’m rewriting some Python code in a low level language, I can’t assume that the two objects are equal because that equality will not remain if either is reassigned. So if I do some operation on one array I have to conform the two arrays for as long as they remain equal, I can’t just do it in one operation because I can’t rely on the objects remaining equal. Is my understanding of this correct? Is there anything I’m missing? Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
For clarification, equality is not identity in Python. e.g. x = 7 y = 7.0 print(x == y) print(x is y) Will return True False Full explanation at https://docs.python.org/3/reference/expressions.html#comparisons From: Python-list on behalf of Chris Angelico Date: Tuesday, January 10, 2023 at 3:47 PM To: Python List Subject: Re: To clarify how Python handles two equal objects *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. *** On Wed, 11 Jan 2023 at 07:41, Jen Kris wrote: > > > Thanks for your comments. I'd like to make one small point. You say: > > "Assignment in Python is a matter of object references. It's not > "conform them as long as they remain equal". You'll have to think in > terms of object references the entire way." > > But where they have been set to the same object, an operation on one will > affect the other as long as they are equal (in Python). So I will have to > conform them in those cases because Python will reflect any math operation in > both the array and the matrix. > It's not that "an operation on one will affect the other" - it's that, no matter how you refer to that object, you're affecting *that one single object*. It's like when you're washing a window; the inside and outside of the window are the exact same window, so regardless of where you're looking at it from, it's the same single window and changes affect it equally. So you shouldn't have to replicate any changes. What should be happening is that you find the right object to mutate, and mutate that. For example: stuff = [[1, 2, 3], [4, 5, 6]] stuff.append(stuff[0]) print(stuff) You now have two references to the same list, inside another list. Any change to stuff[0] is a change to stuff[2], because they're the exact same list. When you append "a reference to this list over here" (which you found by asking for stuff[0]) to the outer list, you get that list. That's Python's object model, and trying to cheat it by copying changes is just going to cause untold nightmares of desynchronization. ChrisA -- https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iORRBJFui8Kn7WzY0ZRPfSdGKcmnDV81UffITsv7ExEAbBXEtv86qC3BOvGaDXCAY708Q4QbDXh0_wo7$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iORRBJFui8Kn7WzY0ZRPfSdGKcmnDV81UffITsv7ExEAbBXEtv86qC3BOvGaDXCAY708Q4QbDXh0_wo7$> -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Wed, 11 Jan 2023 at 07:41, Jen Kris wrote: > > > Thanks for your comments. I'd like to make one small point. You say: > > "Assignment in Python is a matter of object references. It's not > "conform them as long as they remain equal". You'll have to think in > terms of object references the entire way." > > But where they have been set to the same object, an operation on one will > affect the other as long as they are equal (in Python). So I will have to > conform them in those cases because Python will reflect any math operation in > both the array and the matrix. > It's not that "an operation on one will affect the other" - it's that, no matter how you refer to that object, you're affecting *that one single object*. It's like when you're washing a window; the inside and outside of the window are the exact same window, so regardless of where you're looking at it from, it's the same single window and changes affect it equally. So you shouldn't have to replicate any changes. What should be happening is that you find the right object to mutate, and mutate that. For example: stuff = [[1, 2, 3], [4, 5, 6]] stuff.append(stuff[0]) print(stuff) You now have two references to the same list, inside another list. Any change to stuff[0] is a change to stuff[2], because they're the exact same list. When you append "a reference to this list over here" (which you found by asking for stuff[0]) to the outer list, you get that list. That's Python's object model, and trying to cheat it by copying changes is just going to cause untold nightmares of desynchronization. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Thanks for your comments. I'd like to make one small point. You say: "Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way." But where they have been set to the same object, an operation on one will affect the other as long as they are equal (in Python). So I will have to conform them in those cases because Python will reflect any math operation in both the array and the matrix. Jan 10, 2023, 12:28 by ros...@gmail.com: > On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list > wrote: > >> >> I am writing a spot speedup in assembly language for a short but >> computation-intensive Python loop, and I discovered something about Python >> array handling that I would like to clarify. >> >> For a simplified example, I created a matrix mx1 and assigned the array arr1 >> to the third row of the matrix: >> >> mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] >> arr1 = mx1[2] >> >> The pointers to these are now the same: >> >> ida = id(mx1[2]) - 140260325306880 >> idb = id(arr1) - 140260325306880 >> >> That’s great because when I encounter this in assembly or C, I can just >> borrow the pointer to row 3 for the array arr1, on the assumption that they >> will continue to point to the same object. Then when I do any math >> operations in arr1 it will be reflected in both arrays because they are now >> pointing to the same array: >> > > That's not an optimization; what you've done is set arr1 to be a > reference to that object. > >> But on the next iteration we assign arr1 to something else: >> >> arr1 = [ 10, 11, 12 ] >> idc = id(arr1) – 140260325308160 >> idd = id(mx1[2]) – 140260325306880 >> >> Now arr1 is no longer equal to mx1[2], and any subsequent operations in arr1 >> will not affect mx1. >> > > Yep, you have just set arr1 to be a completely different object. > >> So where I’m rewriting some Python code in a low level language, I can’t >> assume that the two objects are equal because that equality will not remain >> if either is reassigned. So if I do some operation on one array I have to >> conform the two arrays for as long as they remain equal, I can’t just do it >> in one operation because I can’t rely on the objects remaining equal. >> >> Is my understanding of this correct? Is there anything I’m missing? >> > > Assignment in Python is a matter of object references. It's not > "conform them as long as they remain equal". You'll have to think in > terms of object references the entire way. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Wed, 11 Jan 2023 at 07:14, Jen Kris via Python-list wrote: > > I am writing a spot speedup in assembly language for a short but > computation-intensive Python loop, and I discovered something about Python > array handling that I would like to clarify. > > For a simplified example, I created a matrix mx1 and assigned the array arr1 > to the third row of the matrix: > > mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] > arr1 = mx1[2] > > The pointers to these are now the same: > > ida = id(mx1[2]) - 140260325306880 > idb = id(arr1) - 140260325306880 > > That’s great because when I encounter this in assembly or C, I can just > borrow the pointer to row 3 for the array arr1, on the assumption that they > will continue to point to the same object. Then when I do any math > operations in arr1 it will be reflected in both arrays because they are now > pointing to the same array: > That's not an optimization; what you've done is set arr1 to be a reference to that object. > But on the next iteration we assign arr1 to something else: > > arr1 = [ 10, 11, 12 ] > idc = id(arr1) – 140260325308160 > idd = id(mx1[2]) – 140260325306880 > > Now arr1 is no longer equal to mx1[2], and any subsequent operations in arr1 > will not affect mx1. Yep, you have just set arr1 to be a completely different object. > So where I’m rewriting some Python code in a low level language, I can’t > assume that the two objects are equal because that equality will not remain > if either is reassigned. So if I do some operation on one array I have to > conform the two arrays for as long as they remain equal, I can’t just do it > in one operation because I can’t rely on the objects remaining equal. > > Is my understanding of this correct? Is there anything I’m missing? > Assignment in Python is a matter of object references. It's not "conform them as long as they remain equal". You'll have to think in terms of object references the entire way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
