[Python-Dev] Re: PEP 558: Defined semantics for locals() (December 2019 edition)

2020-01-01 Thread Nick Coghlan 
On Wed, 1 Jan 2020 at 10:42, Yonatan Zunger  wrote:
>
> Thanks for writing this up, Nick!
>
> My main question is about the remaining difference between semantics at the 
> class/module versus function level: is it worth the additional cognitive 
> complexity to have the class/module behavior be different from the function 
> behavior?

Mutating values through locals() has long worked at class and module
scope, and folks rely on that often enough that we don't want to break
their code. Correctly supporting mutation via locals() also falls
naturally out of the runtime semantics of those scopes (at module
scope, locals() returns the same namespace as globals() does, while at
class scope it returns the same namespace as will eventually be passed
to the metaclass to create the new class object).

By contrast, mutation via locals() at function scope hasn't worked
consistently since fast locals were introduced, and that predates even
statically nested scopes (I went trawling through the repo history to
try to work out when they were first added, and made it as far as
"some time before 1997, since [1] changed how they worked to make them
faster, but was replacing an even older list-based mechanism").

Cheers,
Nick.

[1] 
https://github.com/python/cpython/commit/f3e85a0356e679ed9ff8d13236ff8e9f77a9fd0b#diff-f1de0b7194da5b0cdf48379eca8678d7

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/VATKCBOQKJWFL7TQ5I7NTUHW5OMEYHTY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding a scarier warning to object.__del__?

2020-01-01 Thread Andrew Svetlov 
__del__ is very useful not for interpreter shutdown but as a regular
destructor in object's lifecycle.

Action on the shutdown is another beast.
Personally, I prefer to do all finalization works by explicit calls instead.

On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger  wrote:
>
> Hey everyone,
>
> I just encountered yet another reason to beware of __del__: when it's called 
> during interpreter shutdown, for reasons which are kind of obvious in 
> retrospect, if it calls notify() on a threading.Condition, the waiting thread 
> may or may not ever actually receive it, and so if it does that and then 
> tries to join() the thread the interpreter may hang in a hard-to-debug way.
>
> This isn't something that can reasonably be fixed, and (like in most cases 
> involving __del__) there's a very simple fix of using weakref.finalize 
> instead. My question for the dev list: How would people feel about changing 
> the documentation for the method to more bluntly warn people against using 
> it, and refer them to weakref.finalize and/or atexit.register as an 
> alternative? The text already has an undertone of "lasciate ogni speranza, 
> voi ch'entrate" but it may be helpful to be more explicit to avoid people 
> getting caught in unexpected pitfalls.
>
> Yonatan
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Use of AT_SECURE on Linux

2020-01-01 Thread Max Mazurov 
Hello, list.

I noticed that CPython does not sanitize environment when transition between
SELinux contexts or AppArmor profiles happens, which I think is a good thing to
do in cases when one sandboxed application runs a Python script with different
(potentially less strict) sandbox configuration.

On Linux, such transition is indicated by LSMs (SELinux, AppArmor) using the
AT_SECURE bit in the aux. vector. Consult getauxval(3) in the glibc manual
for details. Here is the paragraph describing AT_SECURE:
> Has a nonzero value if this executable should be treated securely. Most
> commonly, a nonzero value indicates that the process is executing a
> set-user-ID or set-group-ID binary (so that its real and effective UIDs or
> GIDs differ from one another), or that it gained capabilities by executing a
> binary file that has capabilities (see capabilities(7)). Alternatively, a
> nonzero value may be triggered by a Linux Security Module. When this value is
> nonzero, the dynamic linker disables the use of certain environment variables
> (see ld-linux.so(8)) and glibc changes other aspects of its behavior. (See
> also secure_getenv(3).)

set-user-ID and capabilities are not directly relevant to Python since it is
not possible to apply setuid bit or file capabilities to the script (with #!).
Though, it might be relevant for other applications embedding CPython.
The case with LSMs is directly relevant for standalone scripts since both
SELinux and AppArmor allow applying per-script restrictions.

There is the -I flag for the CPython interpreter that in my opinion does 
exactly what should be done when AT_SECURE bit is set. Since isolated 
flag is already checked by all relevant code paths in CPython and 
standard library, I believe it is a good idea to simply make AT_SECURE
imply -I. Though I am not sure whether the behavior for other 
applications embedding CPython should be same.

I can put together a patch to implement that if there is an interest and
somebody can point me to an appropriate place to change.

--
Cheers,
Max Mazurov
https://foxcpp.dev
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/245DOJRXNAD6DCIS4J3EDJHDHVHW5DIF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Skip Montanaro 
I could swear python-dev was older than late April 1999, yet that's as far
back as the MM3 archives go. As evidence, here's an email from Jack Jansen
on 28 April 1999 which was a reply to an earlier message not present in the
current archive:

https://mail.python.org/archives/list/[email protected]/thread/EMH62JYFLIL5FJ3EPOKX3NKCPCO3TCPH/

Any pointers to older messages appreciated...

Skip
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WQGRVOSTETPCN3CK4PMS5XBN67VQRLLN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Nick Coghlan 
On Thu, 2 Jan 2020 at 02:24, Skip Montanaro  wrote:
>
> I could swear python-dev was older than late April 1999, yet that's as far 
> back as the MM3 archives go. As evidence, here's an email from Jack Jansen on 
> 28 April 1999 which was a reply to an earlier message not present in the 
> current archive:
>
> https://mail.python.org/archives/list/[email protected]/thread/EMH62JYFLIL5FJ3EPOKX3NKCPCO3TCPH/
>
> Any pointers to older messages appreciated...

The old MM2 pipermail archives are still online:
https://mail.python.org/pipermail/python-dev/

(That applies to all the migrated m.p.o mailing lists - the old MM2
archives were left alone, with the MM3 archives having new URLs)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TQBUTWMLOI3M2DFOOQEXNJTJRHLLPC4I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Guido van Rossum 
The pipermail archives go back to April 1999. Maybe that's when python-dev
split off python-list? https://mail.python.org/pipermail/python-dev/

Scroll to the end to see April 1999, and it seems python-dev was fresh
then: https://mail.python.org/pipermail/python-dev/1999-April/095127.html

(There's one April 1995 entry, but it appears a misfiled rant from Linus
Torvalds. :-)

On Wed, Jan 1, 2020 at 8:24 AM Skip Montanaro 
wrote:

> I could swear python-dev was older than late April 1999, yet that's as far
> back as the MM3 archives go. As evidence, here's an email from Jack Jansen
> on 28 April 1999 which was a reply to an earlier message not present in the
> current archive:
>
>
> https://mail.python.org/archives/list/[email protected]/thread/EMH62JYFLIL5FJ3EPOKX3NKCPCO3TCPH/
>
> Any pointers to older messages appreciated...
>
> Skip
>
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/WQGRVOSTETPCN3CK4PMS5XBN67VQRLLN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/Z57JLLZFYBHEIMVRL43M7EYLMIOSXB5G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Nick Coghlan 
On Thu, 2 Jan 2020 at 02:42, Nick Coghlan  wrote:
>
> On Thu, 2 Jan 2020 at 02:24, Skip Montanaro  wrote:
> >
> > I could swear python-dev was older than late April 1999, yet that's as far 
> > back as the MM3 archives go. As evidence, here's an email from Jack Jansen 
> > on 28 April 1999 which was a reply to an earlier message not present in the 
> > current archive:
> >
> > https://mail.python.org/archives/list/[email protected]/thread/EMH62JYFLIL5FJ3EPOKX3NKCPCO3TCPH/
> >
> > Any pointers to older messages appreciated...
>
> The old MM2 pipermail archives are still online:
> https://mail.python.org/pipermail/python-dev/

Although, looking at those archives, they're also missing almost
everything from before April 21 1999, so that's not going to answer
your question of where to find the messages from earlier in the 90's.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YFHI4RS7FRAL7KEM6E6TMQWUM2EAARS2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Mark Sapiro 
On 1/1/20 8:42 AM, Nick Coghlan wrote:
> On Thu, 2 Jan 2020 at 02:24, Skip Montanaro  wrote:
>>
>> Any pointers to older messages appreciated...
> 
> The old MM2 pipermail archives are still online:
> https://mail.python.org/pipermail/python-dev/

The problem is the cumulative archive mbox which was imported into
HyperKitty was apparently incomplete. It is actually in pieces which
were concatenated for the import

> -rw-rw-r--   1 mailman mailman   5180827 Jun  4  2019 python-dev-1999.mbox
> -rw-rw-r--   1 mailman mailman  26717370 Jan 18  2010 python-dev-2000.mbox
> -rw-rw-r--   1 mailman mailman  24363693 Jan 18  2010 python-dev-2001.mbox
> -rw-rw-r--   1 mailman mailman  40367530 Jan 18  2010 python-dev-2002.mbox
> -rw-rw-r--   1 mailman mailman  32832292 Jan 18  2010 python-dev-2003.mbox
> -rw-rw-r--   1 mailman mailman  31528859 Jan 18  2010 python-dev-2004.mbox
> -rw-rw-r--   1 mailman mailman  36723436 Jan 18  2010 python-dev-2005.mbox
> -rw-rw-r--   1 mailman mailman  44116711 Jan 18  2010 python-dev-2006.mbox
> -rw-rw-r--   1 mailman mailman  22101735 Jan 18  2010 python-dev-2007.mbox
> -rw-rw-r--   1 mailman mailman  39502442 Jan 18  2010 python-dev-2008.mbox
> -rw-rw-r--   1 mailman mailman  58868434 Jan 18  2010 python-dev-2009.mbox
> -rw-rw-r--   1 mailman mailman 435171834 Jun  4  2019 python-dev.mbox

I'll see about adding the missing messages if they are in the pipermail
archive.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/3QRW3PD4MQ4WIACEPORBAZSJ2TVT2Z7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Barry Warsaw 
I am looking at the MM2 mailing list creation confirmation messages in my 
personal archives.  Both [email protected] (at 09:49 server local time?) and 
[email protected] (at 14:17) were created on April 19, 1999.  I don’t 
remember what happened to dev@ but based on the timeline, I’m retroguessing 
that we created dev@ first, then quickly rethought the name, created 
python-dev@ and retired dev@.

-Barry

> On Jan 1, 2020, at 08:51, Mark Sapiro  wrote:
> 
> On 1/1/20 8:42 AM, Nick Coghlan wrote:
>> On Thu, 2 Jan 2020 at 02:24, Skip Montanaro  wrote:
>>> 
>>> Any pointers to older messages appreciated...
>> 
>> The old MM2 pipermail archives are still online:
>> https://mail.python.org/pipermail/python-dev/
> 
> The problem is the cumulative archive mbox which was imported into
> HyperKitty was apparently incomplete. It is actually in pieces which
> were concatenated for the import
> 
>> -rw-rw-r--   1 mailman mailman   5180827 Jun  4  2019 python-dev-1999.mbox
>> -rw-rw-r--   1 mailman mailman  26717370 Jan 18  2010 python-dev-2000.mbox
>> -rw-rw-r--   1 mailman mailman  24363693 Jan 18  2010 python-dev-2001.mbox
>> -rw-rw-r--   1 mailman mailman  40367530 Jan 18  2010 python-dev-2002.mbox
>> -rw-rw-r--   1 mailman mailman  32832292 Jan 18  2010 python-dev-2003.mbox
>> -rw-rw-r--   1 mailman mailman  31528859 Jan 18  2010 python-dev-2004.mbox
>> -rw-rw-r--   1 mailman mailman  36723436 Jan 18  2010 python-dev-2005.mbox
>> -rw-rw-r--   1 mailman mailman  44116711 Jan 18  2010 python-dev-2006.mbox
>> -rw-rw-r--   1 mailman mailman  22101735 Jan 18  2010 python-dev-2007.mbox
>> -rw-rw-r--   1 mailman mailman  39502442 Jan 18  2010 python-dev-2008.mbox
>> -rw-rw-r--   1 mailman mailman  58868434 Jan 18  2010 python-dev-2009.mbox
>> -rw-rw-r--   1 mailman mailman 435171834 Jun  4  2019 python-dev.mbox
> 
> I'll see about adding the missing messages if they are in the pipermail
> archive.
> 
> --
> Mark Sapiro The highway is for gamblers,
> San Francisco Bay Area, Californiabetter use your sense - B. Dylan



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/WWNVYN64WHNPIHRPY5FES2W2QUTJWUO7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding a scarier warning to object.__del__?

2020-01-01 Thread Gregory P. Smith 
On Wed, Jan 1, 2020 at 6:40 AM Andrew Svetlov 
wrote:

> __del__ is very useful not for interpreter shutdown but as a regular
> destructor in object's lifecycle.
>

The reason we should warn people against ever implementing __del__ is that
people rarely actually understand object lifecycle.  Its presence
guarantees delayed garbage collection and that its code will code execute
in a context way outside of normal control flow that all other methods are
invoked from.

-gps


>
> Action on the shutdown is another beast.
> Personally, I prefer to do all finalization works by explicit calls
> instead.
>
> On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger  wrote:
> >
> > Hey everyone,
> >
> > I just encountered yet another reason to beware of __del__: when it's
> called during interpreter shutdown, for reasons which are kind of obvious
> in retrospect, if it calls notify() on a threading.Condition, the waiting
> thread may or may not ever actually receive it, and so if it does that and
> then tries to join() the thread the interpreter may hang in a hard-to-debug
> way.
> >
> > This isn't something that can reasonably be fixed, and (like in most
> cases involving __del__) there's a very simple fix of using
> weakref.finalize instead. My question for the dev list: How would people
> feel about changing the documentation for the method to more bluntly warn
> people against using it, and refer them to weakref.finalize and/or
> atexit.register as an alternative? The text already has an undertone of
> "lasciate ogni speranza, voi ch'entrate" but it may be helpful to be more
> explicit to avoid people getting caught in unexpected pitfalls.
> >
> > Yonatan
> > ___
> > Python-Dev mailing list -- [email protected]
> > To unsubscribe send an email to [email protected]
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/[email protected]/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Thanks,
> Andrew Svetlov
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TCLYO5W2WL57WV573GMEEVJSA7K6GBMH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding a scarier warning to object.__del__?

2020-01-01 Thread Andrew Svetlov 
If the warning text tells about the delayed execution -- I'm +1.
-1 for something like "just don't use __del__, it is dangerous".

On Wed, Jan 1, 2020, 21:51 Gregory P. Smith  wrote:

>
>
> On Wed, Jan 1, 2020 at 6:40 AM Andrew Svetlov 
> wrote:
>
>> __del__ is very useful not for interpreter shutdown but as a regular
>> destructor in object's lifecycle.
>>
>
> The reason we should warn people against ever implementing __del__ is that
> people rarely actually understand object lifecycle.  Its presence
> guarantees delayed garbage collection and that its code will code execute
> in a context way outside of normal control flow that all other methods are
> invoked from.
>
> -gps
>
>
>>
>> Action on the shutdown is another beast.
>> Personally, I prefer to do all finalization works by explicit calls
>> instead.
>>
>> On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger  wrote:
>> >
>> > Hey everyone,
>> >
>> > I just encountered yet another reason to beware of __del__: when it's
>> called during interpreter shutdown, for reasons which are kind of obvious
>> in retrospect, if it calls notify() on a threading.Condition, the waiting
>> thread may or may not ever actually receive it, and so if it does that and
>> then tries to join() the thread the interpreter may hang in a hard-to-debug
>> way.
>> >
>> > This isn't something that can reasonably be fixed, and (like in most
>> cases involving __del__) there's a very simple fix of using
>> weakref.finalize instead. My question for the dev list: How would people
>> feel about changing the documentation for the method to more bluntly warn
>> people against using it, and refer them to weakref.finalize and/or
>> atexit.register as an alternative? The text already has an undertone of
>> "lasciate ogni speranza, voi ch'entrate" but it may be helpful to be more
>> explicit to avoid people getting caught in unexpected pitfalls.
>> >
>> > Yonatan
>> > ___
>> > Python-Dev mailing list -- [email protected]
>> > To unsubscribe send an email to [email protected]
>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>> > Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/AAZQBWD6PHC4PVNCCPX4A2745SS7B3LS/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>>
>> --
>> Thanks,
>> Andrew Svetlov
>> ___
>> Python-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/57CDW4NIYEQ3JEVX2JVCJDA5TXTC5MBR/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/NFGVEU53S37ZQZ2JEUFQIDYHQ6USR6DG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python-dev mailing lis archives earlier than late April 1999?

2020-01-01 Thread Mark Sapiro 
On 1/1/20 11:22 AM, Barry Warsaw wrote:
> I am looking at the MM2 mailing list creation confirmation messages in my 
> personal archives.  Both [email protected] (at 09:49 server local time?) and 
> [email protected] (at 14:17) were created on April 19, 1999.  I don’t 
> remember what happened to dev@ but based on the timeline, I’m retroguessing 
> that we created dev@ first, then quickly rethought the name, created 
> python-dev@ and retired dev@.

Just to provide some closure here, the pipermail archive for python-dev
goes back to April 21, 1999. There is one, possibly spurious message
from some other list dated March 16, 1995 from Linus Torvalds.

Aside from this one message and as far as I can tell, all the other
messages from April 21 forward are in the current Hyperkitty archive.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/IH4TADHYFDTCKPZNNLGU2PNC23QFDM64/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 558: Defined semantics for locals() (December 2019 edition)

2020-01-01 Thread Yonatan Zunger 
Makes sense. Thanks for the clarification!

On Wed, Jan 1, 2020 at 1:30 AM Nick Coghlan  wrote:

> On Wed, 1 Jan 2020 at 10:42, Yonatan Zunger  wrote:
> >
> > Thanks for writing this up, Nick!
> >
> > My main question is about the remaining difference between semantics at
> the class/module versus function level: is it worth the additional
> cognitive complexity to have the class/module behavior be different from
> the function behavior?
>
> Mutating values through locals() has long worked at class and module
> scope, and folks rely on that often enough that we don't want to break
> their code. Correctly supporting mutation via locals() also falls
> naturally out of the runtime semantics of those scopes (at module
> scope, locals() returns the same namespace as globals() does, while at
> class scope it returns the same namespace as will eventually be passed
> to the metaclass to create the new class object).
>
> By contrast, mutation via locals() at function scope hasn't worked
> consistently since fast locals were introduced, and that predates even
> statically nested scopes (I went trawling through the repo history to
> try to work out when they were first added, and made it as far as
> "some time before 1997, since [1] changed how they worked to make them
> faster, but was replacing an even older list-based mechanism").
>
> Cheers,
> Nick.
>
> [1]
> https://github.com/python/cpython/commit/f3e85a0356e679ed9ff8d13236ff8e9f77a9fd0b#diff-f1de0b7194da5b0cdf48379eca8678d7
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/S3MOFZMX6NWUCNL33ROALUNJDGN3YQ5H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding a scarier warning to object.__del__?

2020-01-01 Thread Yonatan Zunger 
Oh, I'm absolutely thinking about clarity. Something like:



This method is called when the instance is about to be destroyed. Because
it may be called by either the ordinary Python execution flow or by the
garbage collector, it has very unusual semantics, and must be treated with
great care. In almost all cases, you should instead use either
weakref.finalize (if you want logic to be called when an object is no
longer in use) or atexit.register (if you want to clean up before the
interpreter exits).

The precise semantics of when __del__ is called on an object are
implementation-dependent. For example:
* It might be invoked during the normal interpreter flow at a moment like
function return, if the reference count for an object has dropped to zero;
* It might be invoked by the garbage collector, which happens during the
normal interpreter flow but is called from an arbitrary thread;
* It might be invoked during interpreter shutdown, after various global
variables (including modules!) have already been deleted, or other threads'
interpreters have already stopped.
* It may not be invoked at all; it is *not guaranteed* that __del__()
methods are called for objects that still exist when the interpreter exits.

In particular, this means that, unless your program is structured so that
you can deterministically control when an object will be destroyed,
implementations of __del__():
* Must not assume that any variables outside self exist, including modules
or global variables;
* Must not attempt to acquire locks or otherwise block, since they may
(e.g.) be invoked inside a thread which already holds those resources;
* Must not rely on being able to signal other threads and wait for their
responses (even thread joins), since those threads' interpreters may have
exited;
* Must not cause failures (e.g. leaving external devices in an invalid
state) if they are *never* invoked.

Note that del x doesn’t directly call x.__del__() — the former decrements
the reference count for x by one, and the latter is only called when x’s
reference count reaches zero. Depending on the implementation, it is
possible for a reference cycle to prevent the reference count of an object
from going to zero. (e.g., in CPython, a common cause of reference cycles
is when an exception is caught and stored in a local variable; the
exception contains a reference to the traceback, which in turn references
the locals of all frames caught in the traceback.) In this case, the cycle
will be later detected and deleted by the cyclic garbage collector.

If a base class has a __del__() method, the derived class’s __del__()
method, if any, must explicitly call it to ensure proper deletion of the
base class part of the instance.

It is possible (though not recommended!) for the __del__() method to
postpone destruction of the instance by creating a new reference to it.
This is called object resurrection. It is implementation-dependent whether
__del__() is called a second time when a resurrected object is about to be
destroyed; the current CPython implementation only calls it once.



On Wed, Jan 1, 2020 at 12:57 PM Andrew Svetlov 
wrote:

> If the warning text tells about the delayed execution -- I'm +1.
> -1 for something like "just don't use __del__, it is dangerous".
>
> On Wed, Jan 1, 2020, 21:51 Gregory P. Smith  wrote:
>
>>
>>
>> On Wed, Jan 1, 2020 at 6:40 AM Andrew Svetlov 
>> wrote:
>>
>>> __del__ is very useful not for interpreter shutdown but as a regular
>>> destructor in object's lifecycle.
>>>
>>
>> The reason we should warn people against ever implementing __del__ is
>> that people rarely actually understand object lifecycle.  Its presence
>> guarantees delayed garbage collection and that its code will code execute
>> in a context way outside of normal control flow that all other methods are
>> invoked from.
>>
>> -gps
>>
>>
>>>
>>> Action on the shutdown is another beast.
>>> Personally, I prefer to do all finalization works by explicit calls
>>> instead.
>>>
>>> On Wed, Jan 1, 2020 at 2:39 AM Yonatan Zunger  wrote:
>>> >
>>> > Hey everyone,
>>> >
>>> > I just encountered yet another reason to beware of __del__: when it's
>>> called during interpreter shutdown, for reasons which are kind of obvious
>>> in retrospect, if it calls notify() on a threading.Condition, the waiting
>>> thread may or may not ever actually receive it, and so if it does that and
>>> then tries to join() the thread the interpreter may hang in a hard-to-debug
>>> way.
>>> >
>>> > This isn't something that can reasonably be fixed, and (like in most
>>> cases involving __del__) there's a very simple fix of using
>>> weakref.finalize instead. My question for the dev list: How would people
>>> feel about changing the documentation for the method to more bluntly warn
>>> people against using it, and refer them to weakref.finalize and/or
>>> atexit.register as an alternative? The text already has an undertone of
>>> "lasciate ogni speranza, voi ch'entrate" but it may be hel