Hit "Reply" instead of "Reply All" last night, oops. Forwarding to the
list for posterity's sakes.
/
/arry/
Forwarded Message
Subject: Re: [Python-Dev] PyWeakref_GetObject() borrows its reference
from... whom?
Date: Mon, 10 Oct 2016 23:01:10 +0200
From: Larry Hasting
Elliot Gorokhovsky wrote:
I ran the
benchmark a couple of times and the numbers seem to exactly line up
something like one in five times; perhaps not that crazy considering
they're executing nearly the same code?
Could this be a result of a time being measured in seconds
somewhere and then di
On 11 October 2016 at 08:49, Guido van Rossum wrote:
> On Mon, Oct 10, 2016 at 1:57 PM, Larry Hastings wrote:
>> For now I'm going to leave the names as-is and just change the semantics
>> everywhere. If this approach really works, and if we decide we want to
>> merge it back into trunk--and tho
I only mentioned it because the OP mentioned the theme as one of his
reasons to prefer one over the other.
On Mon, Oct 10, 2016 at 8:16 PM, wrote:
>
>
>> -Original Message-
>> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
>> list=sdamon@python.org] On Behalf Of Guido van
> -Original Message-
> From: Distutils-SIG [mailto:distutils-sig-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Guido van Rossum
> Sent: Monday, October 10, 2016 12:27 PM
> To: Donald Stufft
> Cc: Distutils ; Giampaolo Rodola'
> ; python-dev
> Subject: Re: [Distutils] [Pyth
Right - sorry about that last bit!
I couldn't figure out how to use timeit/perf for this because the list has
to be reinitialized each time it gets sorted. So with time.time() I can
just wrap the sorting and cut the initialization out (and I could always
throw it in a for loop to do it as many tim
So here's a simple attempt at taking lots of measurements just using
time.time() with lists of ints. The results are great, if they are valid
(which I leave to you to judge); even for lists with just one element, it's
16% faster! The columns are length, number of trials, and percent
improvement:
I
[please restrict follow-ups to python-ideas]
Let's not get hung up on meta-discussion here - I always thought "massive
clusterf**k" was a precise technical term anyway ;-)
While timing certainly needs to be done more carefully, it's obvious to me
that this approach _should_ pay off significantly
On Mon, Oct 10, 2016 at 3:27 PM, Greg Ewing wrote:
> Nathaniel Smith wrote:
>>
>> IIRC to handle
>> this gilectomy adds per-object mutexes that you have to hold whenever
>> you're mucking around with that object's internals.
>
>
> What counts as "mucking around with the object's internals",
> thou
So maybe someone should explain to Elliott *why* his own benchmarks
are not trustworthy, rather than just repeat "use perf or timeit".
Actually, there are two things: (a) when something new comes along it
*always* needs to prove beyond a shadow of a doubt that it is actually
an improvement and not
On Mon, Oct 10, 2016 at 09:16:32PM +, Elliot Gorokhovsky wrote:
> Anyway, benchmarking technique aside, the point is that it it works well
> for small lists (i.e. doesn't affect performance).
You've been shown that there is something suspicious about your
benchmarking technique, something th
On behalf of the Python development community and the Python 3.6 release
team, I'm happy to announce the availability of Python 3.6.0b2. 3.6.0b2
is the second of four planned beta releases of Python 3.6, the next major
release of Python, and marks the end of the feature development phase
for 3.6.
On Tue, Oct 11, 2016 at 9:52 AM, MRAB wrote:
>> Also, I don't know when it would ever be safe to release the "memory
>> deallocation lock". Just because it's safe for your thread doesn't mean
>> it's safe for another thread. And if you do it on a thread-by-thread
>> basis, in the above example i
On 2016-10-10 22:14, Larry Hastings wrote:
On 10/10/2016 09:36 PM, Chris Angelico wrote:
Hmm. Here's a naughty, and maybe dangerous, theory. Obtain a "memory
deallocation lock". While it is held (by any thread - it's a guard,
more than a lock), Py_DECREF will not actually deallocate memory -
ob
On Mon, Oct 10, 2016 at 1:57 PM, Larry Hastings wrote:
>
> On 10/10/2016 06:37 PM, Guido van Rossum wrote:
>
> Modified +1: you can't change the behavior of the existing API, but
> you can deprecate it and introduce a better one with a different name.
> We'll have until Python 4.0 to carry through
Random832 wrote:
Or is this a special kind of lock that you
can "assert isn't locked" without locking it for yourself, and
INCREF/DECREF does so?
I don't think that would work. It might be unlocked at
the moment you test it, but someone might lock it between
then and the following INCREF/DECREF
Nathaniel Smith wrote:
IIRC to handle
this gilectomy adds per-object mutexes that you have to hold whenever
you're mucking around with that object's internals.
What counts as "mucking around with the object's internals",
though?
If I do the C equivalent of:
x = somedict[5]
x.dosomething
On Tue, Oct 11, 2016 at 4:03 AM, Paul Moore wrote:
If you have an object that forever owns a reference to
another object, it's safe to return borrowed references.
Even if there are such cases, it seems we're going to
have to be a lot more careful about identifying them.
It might be safer not
On Mon, Oct 10, 2016 at 2:16 PM, Elliot Gorokhovsky
wrote:
> Hm... that is strange, but I don't think there's anything wrong with the way
> I'm timing, though I agree perf/timeit would be better. I ran the benchmark
> a couple of times and the numbers seem to exactly line up something like one
> i
On Tue, Oct 11, 2016 at 8:14 AM, Larry Hastings wrote:
> But I don't think this fixes the problem. Consider:
>
> Thread A calls Q = PyList_GetItem(L, 0), which returns a borrowed reference.
> Thread A then gets suspended, before it has a chance to Py_INCREF(Q).
> Thread B does L.clear(), the refe
On Tue, Oct 11, 2016 at 7:42 AM, Elliot Gorokhovsky
wrote:
> ChrisA suggested I also try "make test" or something to get a more realistic
> benchmark. I will do that once I implement this as a patch, right now it's
> an extension module that subclasses list, so I can't just drop it into
> existing
Hm... that is strange, but I don't think there's anything wrong with the
way I'm timing, though I agree perf/timeit would be better. I ran the
benchmark a couple of times and the numbers seem to exactly line up
something like one in five times; perhaps not that crazy considering
they're executing n
On 10/10/2016 09:36 PM, Chris Angelico wrote:
Hmm. Here's a naughty, and maybe dangerous, theory. Obtain a "memory
deallocation lock". While it is held (by any thread - it's a guard,
more than a lock), Py_DECREF will not actually deallocate memory -
objects can fall to zero references without be
On 10/10/2016 06:37 PM, Guido van Rossum wrote:
Modified +1: you can't change the behavior of the existing API, but
you can deprecate it and introduce a better one with a different name.
We'll have until Python 4.0 to carry through the deprecation anyways.
And I doubt this is the only C API chan
On Mon, Oct 10, 2016 at 1:42 PM, Elliot Gorokhovsky
wrote:
> *** 10 strings ***
> F.fastsort(): 1.6689300537109375e-06
> F.sort(): 1.6689300537109375e-06
I think something has gone wrong with your timing harness...
For accurately timing microbenchmarks, you should use timeit, or
better yet Victo
I'd like to reply to both questions with one email, if that's all right.
Bernardo Sulzbach asked, "How does this interfere with sorting very small
lists?", and ChrisA asked, "How much slower are sorts of heterogeneous
lists?". I modified the benchmark to answer both questions, the former at
the to
On 2016-10-10 20:36, Chris Angelico wrote:
On Tue, Oct 11, 2016 at 5:24 AM, Random832 wrote:
On Mon, Oct 10, 2016, at 14:04, MRAB wrote:
Instead of locking the object, could we keep the GIL, but have it
normally released?
A thread could then still call a function such as PyWeakref_GetObject()
On Tue, Oct 11, 2016 at 5:24 AM, Random832 wrote:
> On Mon, Oct 10, 2016, at 14:04, MRAB wrote:
>> Instead of locking the object, could we keep the GIL, but have it
>> normally released?
>>
>> A thread could then still call a function such as PyWeakref_GetObject()
>> that returns a borrowed refere
On Mon, Oct 10, 2016, at 14:04, MRAB wrote:
> Instead of locking the object, could we keep the GIL, but have it
> normally released?
>
> A thread could then still call a function such as PyWeakref_GetObject()
> that returns a borrowed reference, but only if it's holding the GIL. It
> would be a
On 2016-10-10 18:50, Nathaniel Smith wrote:
On Mon, Oct 10, 2016 at 10:03 AM, Paul Moore wrote:
> On 10 October 2016 at 17:49, MRAB wrote:
>> If you lookup something in a dict, it'll be a borrowed reference.
>>
>> If the dict is globals() and there's no GIL, another thread could delete the
>> i
On 10 October 2016 at 18:50, Nathaniel Smith wrote:
> I assume Larry is way ahead of us on this,
Yeah, I'd imagine you're right on that :-)
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubs
On Mon, Oct 10, 2016 at 10:03 AM, Paul Moore wrote:
> On 10 October 2016 at 17:49, MRAB wrote:
>> If you lookup something in a dict, it'll be a borrowed reference.
>>
>> If the dict is globals() and there's no GIL, another thread could delete the
>> item while your code had the borrowed reference
On Tue, Oct 11, 2016 at 4:03 AM, Paul Moore wrote:
> On 10 October 2016 at 17:49, MRAB wrote:
>> If you lookup something in a dict, it'll be a borrowed reference.
>>
>> If the dict is globals() and there's no GIL, another thread could delete the
>> item while your code had the borrowed reference.
On Tue, Oct 11, 2016 at 3:49 AM, MRAB wrote:
> On 2016-10-10 10:45, Chris Angelico wrote:
>>
>> On Mon, Oct 10, 2016 at 8:35 PM, Larry Hastings
>> wrote:
>>>
>>> Huh? In all other circumstances, a "borrowed" reference is exactly that:
>>> X
>>> has a reference, and you are relying on X's referen
On 10 October 2016 at 17:49, MRAB wrote:
> If you lookup something in a dict, it'll be a borrowed reference.
>
> If the dict is globals() and there's no GIL, another thread could delete the
> item while your code had the borrowed reference.
>
> It looks like there might be a lot that will need to
On 2016-10-10 10:45, Chris Angelico wrote:
On Mon, Oct 10, 2016 at 8:35 PM, Larry Hastings wrote:
Huh? In all other circumstances, a "borrowed" reference is exactly that: X
has a reference, and you are relying on X's reference to keep the object
alive. Borrowing from a borrowed reference is s
Modified +1: you can't change the behavior of the existing API, but
you can deprecate it and introduce a better one with a different name.
We'll have until Python 4.0 to carry through the deprecation anyways.
And I doubt this is the only C API change needed for happy gil-free
coding...
On Mon, Oct
Honestly I like readthedocs a lot, and I actually *don't* like docs
that look too much like the standard Python docs -- it should be clear
to readers (subliminally, through page style, not just be parsing the
URL) that they're reading third party docs.
On Mon, Oct 10, 2016 at 9:06 AM, Donald Stuff
You’re using the new PyPI (either by getting a new default from a new version
of Python or Twine OR explicitly switching to it yourself) which has this
disabled. You can still upload to legacy PyPI by switching the default back.
That will restore the ability to upload docs to pythonhosted.org.
On Mon, Oct 10, 2016 at 5:18 PM, Elliot Gorokhovsky
wrote:
> First, some simple benchmark results (numbers are seconds, check out the
> extension module at https://github.com/embg/python-fast-listsort.git):
>
> *** 1e3 ints ***
> F.fastsort(): 0.00018930435180664062
> F.sort(): 0.00028300285339355
On 10/10/2016 03:18 AM, Elliot Gorokhovsky wrote:
Hi,
I posted here a while back asking what you guys thought about
implementing radix sort for strings in list.sort(). You gave me a lot of
reasons why that would be a bad idea. However, it got me thinking, and I
came up with something that you ma
Thanks Paul,
no I wasn't aware of that, and I will subscribe to distutils-sig from now
on.
I don't remember ever receiving a warrant about this decision but it's
entirely possible I may have forgot. =)
So long story short is that I will have to move the doc on readthedocs,
correct?
Does pythonhoste
Hi,
I posted here a while back asking what you guys thought about implementing
radix sort for strings in list.sort(). You gave me a lot of reasons why
that would be a bad idea. However, it got me thinking, and I came up with
something that you may find interesting.
First, some simple benchmark re
On 10 October 2016 at 14:34, Giampaolo Rodola' wrote:
> This is what I've bumped into just now:
>
> python setup.py upload_sphinx --upload-dir=docs/_build/html
> running upload_sphinx
> Submitting documentation to https://upload.pypi.org/legacy/
> Upload failed (410): Uploading documentation is no
This is what I've bumped into just now:
python setup.py upload_sphinx --upload-dir=docs/_build/html
running upload_sphinx
Submitting documentation to https://upload.pypi.org/legacy/
Upload failed (410): Uploading documentation is no longer supported, we
recommend using https://readthedocs.org/.
P
On Mon, Oct 10, 2016 at 4:17 AM, Larry Hastings wrote:
> Given that the weakref doesn't have a reference to the object--merely a weak
> reference, different thing--whose reference is it borrowing?
As others have said, it doesn't really matter who's reference it was;
just that there was another at
> On 2016-10-10, at 11:05 , Devin Jeanpierre wrote:
> The term "borrowed" is supposed to imply a sensible scope during which you're
> free to use the object, and weakrefs don't have that (except for what is
> granted by the GIL), so this does sound wacky. I bet it was for performance.
Especial
Larry Hastings wrote:
In contrast, the "borrowed" reference returned by PyWeakRef_GetObject()
seems to be "borrowed" from some unspecified entity.
It's a delocalised quantum reference, borrowing a little
bit from all strong references in existence. :-)
--
Greg
_
On Mon, Oct 10, 2016 at 8:35 PM, Larry Hastings wrote:
> Huh? In all other circumstances, a "borrowed" reference is exactly that: X
> has a reference, and you are relying on X's reference to keep the object
> alive. Borrowing from a borrowed reference is simply a chain of these; Z
> borrows from
On 10/10/2016 11:05 AM, Devin Jeanpierre wrote:
> whose reference is it borrowing?
I think this is a red herring sort of question. "borrowed" only means
"unowned". But anyway, we borrowed from the weakref. (Borrowing from
somebody doesn't imply they own a reference -- we can borrow from a
b
> It's my contention that this API is simply untenable under the Gilectomy
Yeah, I agree with your contention. The only circumstance PyWeakref_GetObject
would still be valid without a GIL is if you happened to know you had a
reference elsewhere that was keeping it alive. But if you did, you probab
The documentation for PyWeakref_GetObject() states:
Return value: Borrowed reference.
Return the referenced object from a weak reference, ref. If the
referent is no longer live, returns Py_None.
Given that the weakref doesn't have a reference to the object--merely a
weak reference, d
52 matches
Mail list logo
