Re: [Python-Dev] gc ideas -- sparse memory

2010-12-05 Thread Stephen J. Turnbull
Martin v. Löwis writes: Why is useful to expose an identity hash? AFAICS it is *only* useful in building an identity hash table. If so, why not just provide id() or the is operator or both and be done with it? That's precisely James' point: Java provides the identity hash

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-05 Thread Greg Ewing
Martin v. Löwis wrote: The hole C API would break if objects would move in memory. Since they have to stay at fixed addresses, it's easy enough to use the address as ID. Yes. Some of the discussion here seems to be assuming that the reason Python doesn't move objects is so that it can use the

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
Am I still missing something? Apparently. The hole C API would break if objects would move in memory. Since they have to stay at fixed addresses, it's easy enough to use the address as ID. There is no problem to be solved here. Regards, Martin ___

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
I'm afraid I don't follow you. Unless you're suggesting some sort of esoteric object system whereby objects *don't* have identity (e.g. where objects are emergent properties of some sort of distributed, non-localised information), any object naturally has an identity -- itself. Not in Java

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Steven D'Aprano
Martin v. Löwis wrote: I'm afraid I don't follow you. Unless you're suggesting some sort of esoteric object system whereby objects *don't* have identity (e.g. where objects are emergent properties of some sort of distributed, non-localised information), any object naturally has an identity --

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Antoine Pitrou
On Sat, 4 Dec 2010 15:06:45 +1100 Cameron Simpson c...@zip.com.au wrote: On 03Dec2010 18:15, James Y Knight f...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython uses ref counting + cycle | gc. A constraint on all implementations is

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
Surely even in Java or C#, objects have an *identity* even if the language doesn't provide a way to query their distinct *identification*. When people said the id of an object should this or that, they always meant identification, not identity (perhaps except for you). Nobody would propose that

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Stephen J. Turnbull
Steven D'Aprano writes: Martin v. Löwis wrote: It seems counter-productive to me to bother with an identity function which doesn't meet that constraint. If id(x) == id(y) implies nothing about x and y (they may, or may not, be the same object) then what's the point? See

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Maciej Fijalkowski
On Sat, Dec 4, 2010 at 1:37 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 4 Dec 2010 15:06:45 +1100 Cameron Simpson c...@zip.com.au wrote: On 03Dec2010 18:15, James Y Knight f...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Stephen J. Turnbull
Quotes are out of order. Martin v. Löwis writes: No, it's not. java.lang.Object.hashCode() is equivalent to Python's hash(). In particular, for both of these, the following requirement holds: object that *compare* equal must hash equal. Aha. I see, now. That is indeed a different

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
Why is useful to expose an identity hash? AFAICS it is *only* useful in building an identity hash table. If so, why not just provide id() or the is operator or both and be done with it? That's precisely James' point: Java provides the identity hash *instead* of the id() function (i.e. it

[Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
How hard or reasonable would it be to free memory pages on OS level? [pcmiiw] Gabage collection within a generation involves moving live objects to compact the generation storage. This separates the memory region into 2 parts live and cleared, the pointer to the beginning of the cleared part is

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 5:55 PM, Dima Tisnek wrote: How hard or reasonable would it be to free memory pages on OS level? [pcmiiw] Gabage collection within a generation involves moving live objects to compact the generation storage. This separates the memory region into 2 parts live and cleared, the

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: gc is implementation specific. CPython uses ref counting + cycle gc. A constraint on all implementations is that objects have a fixed, unique id during their lifetime. CPython uses the address as the id, so it cannot move objects. Other

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Martin v. Löwis
Am 03.12.2010 23:55, schrieb Dima Tisnek: How hard or reasonable would it be to free memory pages on OS level? Very easy. Python already does that. [pcmiiw] Gabage collection within a generation involves moving live objects to compact the generation storage. This separates the memory region

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
Oh my bad, I must've confused python with some research paper. Unique id is not so hard to make without an address. While on this topic, what is the real need for unique ids? Also I reckon not all objects need a unique id like this, e.g. interned strings, simple data types and hashable and

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Martin v. Löwis
Oh my bad, I must've confused python with some research paper. Unique id is not so hard to make without an address. While on this topic, what is the real need for unique ids? They are absolutely needed for mutable objects. For immutable ones, it would be ok to claim that they are identical

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 6:15 PM, James Y Knight wrote: On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: gc is implementation specific. CPython uses ref counting + cycle gc. A constraint on all implementations is that objects have a fixed, unique id during their lifetime. CPython uses the address as the id,

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 7:05 PM, Terry Reedy wrote: I left out that the id must be an int. It's somewhat unfortuante that python has this constraint, instead of the looser: objects have a fixed id during their lifetime, which is much easier to implement, and practically as useful. Given that

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using identityHashCode()), you can still make an identity I do not see the point of calling a (non-unique) hash value

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Cameron Simpson
On 03Dec2010 18:15, James Y Knight f...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython uses ref counting + cycle | gc. A constraint on all implementations is that objects have a fixed, | unique id during their lifetime. CPython uses the

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using identityHashCode()), you can still make an identity I do

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 11:06 PM, Cameron Simpson wrote: On 03Dec2010 18:15, James Y Knightf...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython uses ref counting + cycle | gc. A constraint on all implementations is that objects have a fixed, |

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
On 3 December 2010 16:45, Martin v. Löwis mar...@v.loewis.de wrote: Oh my bad, I must've confused python with some research paper. Unique id is not so hard to make without an address. While on this topic, what is the real need for unique ids? They are absolutely needed for mutable objects.

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Maciej Fijalkowski
On Sat, Dec 4, 2010 at 6:34 AM, James Y Knight f...@fuhm.net wrote: On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Steven D'Aprano
James Y Knight wrote: On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using identityHashCode()), you can still make