[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: I've implemented a new revision that works without making slices hashable. Updating PR 23496 -- ___ Python tracker ___

[issue42454] Move slice creation to the compiler for constants

Andrei Kulakov added the comment: One possibility for this being a breaking change is this scenario: some codebase may have logic that takes a list and uses a slice operation on it; in a rare circumstance the list is really a dict and a TypeError is caught upstream and dealt with; with this

[issue42454] Move slice creation to the compiler for constants

Mark Dickinson added the comment: @Batuhan Taskaya: would it be worth starting a discussion on either python-ideas or python-dev? (Or on discuss.python.org.) My concern is that we're discussing a core language change. It's not a major change, but it's not an insignificant one either, and

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: I've given this another shot, but even though I was able to create a patch that preserved slices as (type(slice), start, stop, step) tuples in the consts_cache, this final optimization prevented me to finalize it;

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: > I have to say that I feel that this will complicate things too much. Part of > the appeal of this change is how straightforward and maintainable is. > Fiddling with code objects lowers the watermark. Agreed. And I'd go for either making slices hashable

[issue42454] Move slice creation to the compiler for constants

Pablo Galindo Salgado added the comment: > I thought about using code.__hash__ in the first place, but the compiler's > underlying store system (both compiler_unit->u_consts and > compiler->c_const_cache) uses dictionary's where the keys are constant > objects themselves (or tuples where

[issue42454] Move slice creation to the compiler for constants

Josh Rosenberg added the comment: Yep, Mark Shannon's solution of contextual hashing is what I was trying (without success) when my last computer died (without backing up work offsite, oops) and I gave up on this for a while. And Batuhan Taskaya's note about compiler dictionaries for the

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: > By leaving slices unhashable, and accounting for their presence in > code.__hash__, we get both the performance improvement and full backwards > compatibility. I thought about using code.__hash__ in the first place, but the compiler's underlying store

[issue42454] Move slice creation to the compiler for constants

Mark Shannon added the comment: I agree with Mark about slice hashing. This looks like a deliberate design decision. x[:] = [] should empty a sequence, not set the key-value pair (slice(None, None, None), []) in a mapping. However, code-objects can still be hashable even if slices are not.

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: One thing to add, the proposed implementation also optimizes extended slices (mostly used in scientific stacks) such as [:, 1]. -- ___ Python tracker

[issue42454] Move slice creation to the compiler for constants

Terry J. Reedy added the comment: I closed #11107 in favor of this issue. Some of the discussion and concerns (like slice hashing) was similar. -- nosy: +terry.reedy ___ Python tracker

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: > What are the potential benefits or drawbacks for the user? The only potential drawback that I can see is that it prevents you from distinguishing a sequence from mapping by 'accidentally' passing a slice. The major benefit for users is that they will

[issue42454] Move slice creation to the compiler for constants

Mark Dickinson added the comment: > At least for optimization, IMHO it worth taking the shot. For me, this feels a bit backwards: IMO you should decide what behaviour you want first, implement the desired behaviour, and then optimize (if possible) while keeping that same desired behaviour.

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: > something[::-1] = something I was thinking this for a while, and this is the actual reason that I didn't propose this until I saw someone's comment under Raymond's tweet about 'slices are not hashable'

[issue42454] Move slice creation to the compiler for constants

Josh Rosenberg added the comment: There is an open issue for this already, under #11107 (a reopen of the closed #2268, where the reopen was justified due to Python 3 making slice objects more common), just so you know. I made a stab at this a while ago and gave up due to the problems with

[issue42454] Move slice creation to the compiler for constants

Raymond Hettinger added the comment: Mostly, +1 from me as well. If anything starts to rely on hashability, it will need a fallback path since slice objects are allowed to contain arbitrary objects (which might not themselves be hashable): s = slice([], []). --

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: As a side effect, we also now fold some esoteric cases which are not very common. Like; """ test """[1:-1] We reviewed only optimizing this a while back and decided these are not that common to add a code path. Just wanted to mention that now we optimize

[issue42454] Move slice creation to the compiler for constants

Christian Heimes added the comment: Good point and excellent explanation. I'm no longer concerned! :) -- ___ Python tracker ___

[issue42454] Move slice creation to the compiler for constants

Batuhan Taskaya added the comment: > I'm slightly concerned about hashability of slice objects. Currently the > slice constructor does not ensure that any slice parameter is a number. You > can do horrible stuff like this: The same thing can be applied to tuples, which are hashable if all

[issue42454] Move slice creation to the compiler for constants

Christian Heimes added the comment: I'm slightly concerned about hashability of slice objects. Currently the slice constructor does not ensure that any slice parameter is a number. You can do horrible stuff like this: >>> slice({}) slice(None, {}, None) which of course fails later on: >>>

[issue42454] Move slice creation to the compiler for constants

Pablo Galindo Salgado added the comment: Unless we see something fundamentally broken with the hashability, I am +1 on this. Even if it does not show in macro benchmarks over the 3% mark, the microbenchmarks are positive and the code changes are very scoped, so there is not a

[issue42454] Move slice creation to the compiler for constants

Change by Raymond Hettinger : -- nosy: +mark.dickinson, rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue42454] Move slice creation to the compiler for constants

Change by Batuhan Taskaya : -- nosy: +Mark.Shannon, serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue42454] Move slice creation to the compiler for constants

Change by Batuhan Taskaya : -- keywords: +patch pull_requests: +22386 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23496 ___ Python tracker ___

[issue42454] Move slice creation to the compiler for constants

New submission from Batuhan Taskaya : Currently, all the slices are created by the interpreter with BUILD_SLICE preceded by 2/3 LOAD_CONSTS. We can move the slice creation into the compiler and deduce the cost of these 3/4 instructions into a single LOAD_CONST operation where all values in