On Wed, 11 Mar 2020 at 18:08, Serhiy Storchaka wrote:
> There is a precedence (although not in the stdlib): NumPy array with
> dtype=bool.
>
> >>> import numpy
> >>> bool(numpy.array([False, False]))
> Traceback (most recent call last):
> File "", line 1, in
> ValueError: The truth v
TL;DR: should we make `del x` an expression that returns the value of `x`.
## Motivation
I noticed yesterday that `itertools.combinations` has an optimization for when
the returned tuple has no remaining ref-counts, and reuses it - namely, the
following code:
>>> for v in itertools.combina
I can be wrong, but for what I know, `del variable_name` does not
optimize the code, in the sense of performance. On the contrary, it
should slow it down, since it simply removes the binding between the
variable name from the namespace, allowing the gc to free the memory
if that variable was the la
On Thu, 12 Mar 2020 at 15:06, Eric Wieser wrote:
> will print the same id three times. However, when used as a list
> comprehension, the optimization can't step in, and I have no way of using the
> `del` keyword
>
> >>> [id(v) for v in itertools.combinations([1, 2, 3], 1)]
> [2500926200
Marco Sulla wrote:
> I can be wrong, but for what I know, del variable_name does not
> optimize the code, in the sense of performance.
Correct, by itself `del variable_name` does not optimize the code - however,
there exist functions implemented in C (which I gave examples of) with special
cases
> You can get the same effect by not naming the value being thrown away
This is absolutely true, although I don't think that's a particularly strong
argument against it. The same can be said of `std::move` in C++. The purpose
of this suggestion is primarily to allow introducing names without it
On Thu, Mar 12, 2020 at 5:50 AM Marco Sulla via Python-ideas <
python-ideas@python.org> wrote:
> Actually, this is the behaviour of ndarray with any dtype. And IMHO
> ithis is quite terrible?
I can see how you would think that. But the fact is that element-wise
operations are very important
It looks like actually this can be be built as a function today:
def move(name):
return inspect.currentframe().f_back.f_locals.pop(name)
Which works as follows, but it feels awkward to pass variable names by strings
(and will confuse linters):
>>> for v in itertools.combinations
On Thu, 12 Mar 2020 at 16:42, Eric Wieser wrote:
> > Do you have an example of an application where this sort of
> > micro-optimisation is significant enough to justify a fairly major language
> > change?
>
> As an example of these optimizations being valuable, see
> https://github.com/numpy/n
On Thu, 12 Mar 2020 at 16:57, Eric Wieser wrote:
>
> It looks like actually this can be be built as a function today:
>
> def move(name):
> return inspect.currentframe().f_back.f_locals.pop(name)
>
> Which works as follows, but it feels awkward to pass variable names by
> strings (and
On Fri, Mar 13, 2020 at 4:04 AM Paul Moore wrote:
>
> On Thu, 12 Mar 2020 at 16:42, Eric Wieser wrote:
>
> > > Do you have an example of an application where this sort of
> > > micro-optimisation is significant enough to justify a fairly major
> > > language change?
> >
> > As an example of the
On Thu, 12 Mar 2020 at 17:09, Christopher Barker wrote:
>> I was so used to have False for an empty iterable
> I hate to be pedantic, but it doesn't work for iterables anyway:
Yes, I know that it does not work for iterators. This is another problem :-)
> but If I really want to know if a sequen
On Thu, Mar 12, 2020 at 03:02:28PM -, Eric Wieser wrote:
> TL;DR: should we make `del x` an expression that returns the value of `x`.
>
> ## Motivation
>
> I noticed yesterday that `itertools.combinations` has an optimization
> for when the returned tuple has no remaining ref-counts, and re
On Mar 12, 2020, at 08:08, Eric Wieser wrote:
>
> TL;DR: should we make `del x` an expression that returns the value of `x`.
I agree that a “move” would have to be a keyword rather than a function, and
that adding a new keyword is too high a bar, and that “del” is the best option
of the exist
On Wed, Mar 11, 2020 at 3:20 AM Kyle Stanley wrote:
> IIUC, I don't think there's a way that we could reasonably support
> stability for provide any guarantees for spawning or interacting with the
> Executors from a daemon thread. It might even be worth considering to
> explicitly warn or prevent
On Thu, 12 Mar 2020 at 18:19, Chris Angelico wrote:
> No, it wouldn't - the use of the value as a return value counts as a
> reference. It's exactly the same as any other function that returns a
> brand-new value.
So the memory of that object will never free... since there's a
reference that can'
On Mar 12, 2020, at 08:53, Eric Wieser wrote:
>
> The nth reference is not removed until the n+1th reference has been created.
> This is unavoidable in a for loop without breaking existing code,
First, is it unavoidable? What could it break if the loop unbound the variable
before nexting the ite
On Fri, Mar 13, 2020 at 4:39 AM Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 18:19, Chris Angelico wrote:
> > No, it wouldn't - the use of the value as a return value counts as a
> > reference. It's exactly the same as any other function that returns a
> > brand-new value.
>
> So the memory of t
On Mar 12, 2020, at 10:52, Marco Sulla via Python-ideas
wrote:
>
> On Thu, 12 Mar 2020 at 18:19, Chris Angelico wrote:
>> No, it wouldn't - the use of the value as a return value counts as a
>> reference. It's exactly the same as any other function that returns a
>> brand-new value.
>
> So th
On Thu, 12 Mar 2020 at 17:42, Eric Wieser wrote:
> As an example of these optimizations being valuable, see
> https://github.com/numpy/numpy/pull/7997, which claims the optimization I
> described at the beginning resulted in a 1.5x-2x speedup.
This speedup is observed only for large objects. I
On Fri, Mar 13, 2020 at 5:16 AM Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 19:05, Chris Angelico wrote:
> > The action of deleting a *name* is not the same as disposing of an
> > *object*. You can consider "del x" to be very similar to "x = None",
> > except that instead of rebinding to some o
On Mar 12, 2020, at 11:23, Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 19:10, Andrew Barnert
> wrote:
>> No, because the return value only lives until (effectively) the end of the
>> statement. A statement has no value, so the effect of an expression
>> statement is to immediately discard
On Thu, Mar 12, 2020 at 10:43 AM Andrew Barnert via Python-ideas <
python-ideas@python.org> wrote:
> What if a for loop, instead of nexting the iterator and binding the result
> to the loop variable, instead unbound the loop variable, nexted the
> Iterator, and bound the result to the loop variabl
On Thu, 12 Mar 2020 at 19:05, Chris Angelico wrote:
> The action of deleting a *name* is not the same as disposing of an
> *object*. You can consider "del x" to be very similar to "x = None",
> except that instead of rebinding to some other object, it unbinds x
> altogether.
Exactly. But if `x =
On Thu, 12 Mar 2020 at 19:10, Andrew Barnert
wrote:
> No, because the return value only lives until (effectively) the end of the
> statement. A statement has no value, so the effect of an expression statement
> is to immediately discard whatever the value of the expression was. (In
> CPython th
On Thu, 12 Mar 2020 at 19:52, Andrew Barnert
wrote:
> I suppose it could track calls out to C functions as they happen and mark
> every value that was live before the call, and then instead of numpy checking
> refs==1 is could check refs==1 && !c_flagged, and then it wouldn’t need the C
> frame
This was exactly what I was thinking of when I said
> This is unavoidable in a for loop without breaking existing code, but I think
> could (and
should?) be changed in a list comprehension
Unfortunately, I was wrong in the last half of this statement.
> since the comprehension control variable
> And as I understand it (from a quick scan) the reason it can’t tell isn’t
> that the refcount isn’t 1 (which is something CPython could maybe fix if it
> were a problem, but it isn’t). Rather, it is already 1, but a refcount of 1
> doesn’t actually prove a temporary value
This is at odds with
On Mar 12, 2020, at 12:02, Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 19:52, Andrew Barnert
> wrote:
>> I suppose it could track calls out to C functions as they happen and mark
>> every value that was live before the call, and then instead of numpy
>> checking refs==1 is could check refs
On Thu, 12 Mar 2020 at 17:20, Chris Angelico wrote:
> > What is to stop a Python implementation (not CPython, maybe, but some
> > other one) from discarding the object in x as soon as it's
> > unreferenced by the (del x) expression? Then y wouldn't be able to
> > hold that value. After all, in the
On Mar 12, 2020, at 12:33, Eric Wieser wrote:
>
>> And as I understand it (from a quick scan) the reason it can’t tell isn’t
>> that the refcount isn’t 1 (which is something CPython could maybe fix if it
>> were a problem, but it isn’t). Rather, it is already 1, but a refcount of 1
>> doesn’t
On 12.03.20 16:02, Eric Wieser wrote
`itertools.combinations` is not the only place to make this optimization -
parts of numpy use it too, allowing
a = (b * c) + d
to elide the temporary `b*c`. This elision can't happen with the spelling
bc = b * c
a = bc + d
This is not surp
On Fri, Mar 13, 2020 at 6:53 AM Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 19:35, Chris Angelico wrote:
> > I don't understand your point.
>
> Yes, Andrew Barnert already explained me that :)
>
> > The broad idea of "del x" returning a value isn't inherently
> > ridiculous
>
> The point is Eri
On Thu, 12 Mar 2020 at 18:42, Andrew Barnert via Python-ideas
wrote:
> What if a for loop, instead of nexting the iterator and binding the result to
> the loop variable, instead unbound the loop variable, nexted the Iterator,
> and bound the result to the loop variable?
I missed that. But I do
On Thu, 12 Mar 2020 at 19:35, Chris Angelico wrote:
> I don't understand your point.
Yes, Andrew Barnert already explained me that :)
> The broad idea of "del x" returning a value isn't inherently
> ridiculous
The point is Eric Wieser does not really want this. What he want is
something similar
On Thu, 12 Mar 2020 at 21:22, Chris Angelico wrote:
> They actually ARE already discarded
0O
You're right. So *how* can juliantaylor said he measured a speedup of
2x for large ndarrays? He added also benchmarks, that are still in
numpy code.
Furthermore he stated that what he done it's done b
On Mar 12, 2020, at 14:32, Marco Sulla via Python-ideas
wrote:
> On Thu, 12 Mar 2020 at 21:22, Chris Angelico wrote:
>> They actually ARE already discarded
>
> 0O
> You're right. So *how* can juliantaylor said he measured a speedup of
> 2x for large ndarrays?
Because Julian’s patch has no
On Thu, 12 Mar 2020 at 21:10, Dominik Vilsmeier
wrote:
If I wanted to split the computation over multiple lines and yet have it
optimized I would just reuse the same (target) name instead of creating
a temporary one and then discarding it in the next step:
a = b * c
a += d
This is
On Thu, Mar 12, 2020 at 10:43 AM Andrew Barnert via Python-ideas
wrote:
> Also, you need to think through what happens with a del expression inside all
> kinds of contexts that currently can’t have del—eval, lambdas and
> comprehensions (can I del something from the outer scope?), etc.; just
>
On Thu, Mar 12, 2020 at 2:32 PM Marco Sulla via Python-ideas
wrote:
>
> On Thu, 12 Mar 2020 at 21:22, Chris Angelico wrote:
> > They actually ARE already discarded
>
> 0O
> You're right. So *how* can juliantaylor said he measured a speedup of
> 2x for large ndarrays?
I think that currently y
Little test:
>>> from itertools import combinations
>>> [id(v) for v in combinations(range(10), 1)]
[140347856824784, 140347856101328, 140347856824784, 140347856101328,
140347856824784, 140347856101328, 140347856824784, 140347856101328,
140347856824784, 140347856101328]
>>> for v in combinations(r
On Mar 12, 2020, at 13:22, Marco Sulla
wrote:
>
> On Thu, 12 Mar 2020 at 18:42, Andrew Barnert via Python-ideas
> wrote:
>> What if a for loop, instead of nexting the iterator and binding the result
>> to the loop variable, instead unbound the loop variable, nexted the
>> Iterator, and bound
???
We are saying the same thing.
On Thu, Mar 12, 2020 at 11:18 AM Marco Sulla via Python-ideas
wrote:
> This speedup is observed only for large objects
On Fri, 13 Mar 2020 at 00:45, Ben Rudiak-Gould wrote:
> it's only beneficial to try it when the arrays are large.
On Thu, 12 Mar 2020 at 23:55, Andrew Barnert
wrote:
> Because Julian’s patch has nothing to do with discarding the temporary
> references. It has to do with knowing that an array is safe to reuse.
Sorry, but I do not understand. Are you saying that the patch does not
create a new temporary ndarr
Well, so all this discussion is only for freeing _one_ memory location earlier?
Seriously... as the other users already said, if someone really need
it, he can use normal loops instead of comprehensions and split
complex expression so temporary objects are immediately free.
Furthermore, 2x speedup
> On Mar 12, 2020, at 17:47, Marco Sulla
> wrote:
>
> Well, so all this discussion is only for freeing _one_ memory location
> earlier?
Basically, yes.
And now I get why you were confused—you’re not confused about Python, you’re
confused about the proposal, because you expected that there m
On 13/03/20 4:02 am, Eric Wieser wrote:
For consistency, `x = (del foo.attr)` and `x = (del foo[i])` could
also become legal expressions, and `__delete__`, `__delattr__`, and
`__delitem__` would now have return values. Existing types would be
free to continue to return `None`.
That would mean
47 matches
Mail list logo
