Re: [Python-Dev] Intricacies of calling __eq__

2014-03-21 Thread Maciej Fijalkowski
On Wed, Mar 19, 2014 at 11:43 PM, Nick Coghlan ncogh...@gmail.com wrote: On 20 Mar 2014 07:38, Nick Coghlan ncogh...@gmail.com wrote: Correct, but I think this discussion has established that how many times dict lookup calls __eq__ on the key is one such thing. In CPython, it already varies

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-21 Thread Nick Coghlan
On 22 March 2014 04:48, Maciej Fijalkowski fij...@gmail.com wrote: at the end of the day we settled for dicts with str int or identity keys, so we're perfectly safe Ah, Armin's original investment in PyPy's type tracking infrastructure pays off yet again :) Cheers, Nick. -- Nick Coghlan |

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Mark Shannon
On 18/03/14 07:52, Maciej Fijalkowski wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup (the second one being constant folded away). The

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Maciej Fijalkowski
On Tue, Mar 18, 2014 at 11:19 PM, Paul Moore p.f.mo...@gmail.com wrote: On 18 March 2014 19:46, Maciej Fijalkowski fij...@gmail.com wrote: A question: how far away will this optimization apply? if x in d: do_this() do_that() do_something_else() spam =

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Antoine Pitrou
On Tue, 18 Mar 2014 09:52:05 +0200 Maciej Fijalkowski fij...@gmail.com wrote: We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup (the second one being constant folded away). The question is whether it's

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Maciej Fijalkowski
On Wed, Mar 19, 2014 at 2:42 PM, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 18 Mar 2014 09:52:05 +0200 Maciej Fijalkowski fij...@gmail.com wrote: We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Hrvoje Niksic
On 03/18/2014 10:19 PM, Paul Moore wrote: Surely in the presence of threads the optimisation is invalid anyway Why? As written, the code uses no synchronization primitives to ensure that the modifications to the dict are propagated at a particular point. As a consequence, it cannot rely on

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Antoine Pitrou
On Wed, 19 Mar 2014 15:09:04 +0200 Maciej Fijalkowski fij...@gmail.com wrote: I would like to point out that instructing people does not really work. Besides, other examples like this: if d[x] = 3: d[x] += 1 don't really work. That's a good point. But then, perhaps PyPy should analyze

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Maciej Fijalkowski
On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 15:09:04 +0200 Maciej Fijalkowski fij...@gmail.com wrote: I would like to point out that instructing people does not really work. Besides, other examples like this: if d[x] = 3: d[x] += 1

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Thomas Wouters
On Wed, Mar 19, 2014 at 6:26 AM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 15:21:16 +0200 Maciej Fijalkowski fij...@gmail.com wrote: On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 15:09:04 +0200 Maciej Fijalkowski

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Antoine Pitrou
On Wed, 19 Mar 2014 07:09:23 -0700 Thomas Wouters tho...@python.org wrote: He means you're being unrealistically pedantic :) The number of calls to __eq__ is _already_ unpredictable, since (as Mark Shannon said) it depends among other things on the hashing algorithm and the size of the dict.

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Antony Lee
The docs don't seem to make any guarantee about calling __eq__ or __hash__: d[key] Return the item of d with key key. Raises a KeyError if key is not in the map. which seems to indicate that this kind of optimization should be fine. In fact I would very much like messing with the semantics of

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Kevin Modzelewski
Sorry, I definitely didn't mean to imply that this kind of optimization is valid on arbitrary subscript expressions; I thought we had restricted ourselves to talking about builtin dicts. If we do, I think this becomes a discussion about what subset of the semantics of CPython's builtins are

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Maciej Fijalkowski
On Wed, Mar 19, 2014 at 3:26 PM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 15:21:16 +0200 Maciej Fijalkowski fij...@gmail.com wrote: On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 19 Mar 2014 15:09:04 +0200 Maciej Fijalkowski

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Maciej Fijalkowski
On Wed, Mar 19, 2014 at 8:38 AM, Kevin Modzelewski k...@dropbox.com wrote: Sorry, I definitely didn't mean to imply that this kind of optimization is valid on arbitrary subscript expressions; I thought we had restricted ourselves to talking about builtin dicts. If we do, I think this becomes a

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Stephen J. Turnbull
Kevin Modzelewski writes: Sorry, I definitely didn't mean to imply that this kind of optimization is valid on arbitrary subscript expressions; I thought we had restricted ourselves to talking about builtin dicts. Ah, maybe so -- Maciej made that clear later for PyPy. My bad. (With the

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Nick Coghlan
On 20 Mar 2014 02:37, Stephen J. Turnbull step...@xemacs.org wrote: Kevin Modzelewski writes: Sorry, I definitely didn't mean to imply that this kind of optimization is valid on arbitrary subscript expressions; I thought we had restricted ourselves to talking about builtin dicts. Ah,

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-19 Thread Nick Coghlan
On 20 Mar 2014 07:38, Nick Coghlan ncogh...@gmail.com wrote: Correct, but I think this discussion has established that how many times dict lookup calls __eq__ on the key is one such thing. In CPython, it already varies based on: - dict contents (due to the identity check and the distribution

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Nick Coghlan
On 18 March 2014 17:52, Maciej Fijalkowski fij...@gmail.com wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup (the second one being

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Maciej Fijalkowski
On Tue, Mar 18, 2014 at 11:35 AM, Nick Coghlan ncogh...@gmail.com wrote: On 18 March 2014 17:52, Maciej Fijalkowski fij...@gmail.com wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] where d is

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Steven D'Aprano
On Tue, Mar 18, 2014 at 05:05:56AM -0400, Terry Reedy wrote: On 3/18/2014 3:52 AM, Maciej Fijalkowski wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] if d.__contains__(x): return

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Maciej Fijalkowski
On Tue, Mar 18, 2014 at 1:18 PM, Steven D'Aprano st...@pearwood.info wrote: On Tue, Mar 18, 2014 at 05:05:56AM -0400, Terry Reedy wrote: On 3/18/2014 3:52 AM, Maciej Fijalkowski wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Steven D'Aprano
On Tue, Mar 18, 2014 at 01:21:05PM +0200, Maciej Fijalkowski wrote: note that this is specifically about dicts, where __eq__ will be called undecided number of times anyway (depending on collisions in hash/buckets which is implementation specific to start with) Exactly. Using a __eq__ method

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/18/2014 07:18 AM, Steven D'Aprano wrote: Nevertheless, an __eq__ with side-effects is legal Python and may in fact be useful. E.g., for an LRU usecase. Tres. - -- === Tres

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Maciej Fijalkowski
On Tue, Mar 18, 2014 at 4:21 PM, Steven D'Aprano st...@pearwood.info wrote: On Tue, Mar 18, 2014 at 01:21:05PM +0200, Maciej Fijalkowski wrote: note that this is specifically about dicts, where __eq__ will be called undecided number of times anyway (depending on collisions in hash/buckets

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Paul Moore
On 18 March 2014 19:46, Maciej Fijalkowski fij...@gmail.com wrote: A question: how far away will this optimization apply? if x in d: do_this() do_that() do_something_else() spam = d[x] it depends what those functions do. JIT will inline them and if

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Kevin Modzelewski
My 2 cents: it feels like a slippery slope to start guaranteeing the number and ordering of calls to comparison functions -- for instance, doing that for the sort() function would lock in the sort implementation. It feels like the number/ordering of the calls should be implementation-defined in

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Steven D'Aprano
On Tue, Mar 18, 2014 at 04:42:29PM -0700, Kevin Modzelewski wrote: My 2 cents: it feels like a slippery slope to start guaranteeing the number and ordering of calls to comparison functions -- for instance, doing that for the sort() function would lock in the sort implementation. Although I

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Kevin Modzelewski
I think in this case, though, if we say for the sake of argument that the guaranteed semantics of a dictionary lookup are zero or more calls to __hash__ plus zero or more calls to __eq__, then two back-to-back dictionary lookups wouldn't have any observable differences from doing only one, unless

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Nick Coghlan
On 19 March 2014 11:09, Steven D'Aprano st...@pearwood.info wrote: Although I have tentatively said I think this is okay, it is a change in actual semantics of Python code: what you write is no longer what gets run. That makes this *very* different from changing the implementation of sort --

Re: [Python-Dev] Intricacies of calling __eq__

2014-03-18 Thread Stephen J. Turnbull
Kevin Modzelewski writes: I think in this case, though, if we say for the sake of argument that the guaranteed semantics of a dictionary lookup are zero or I don't understand the point of that argument. It's simply false that semantics are guaranteed, and all of the dunders might be user