[Python-Dev] Re: [python-committers] Sad news from Zurich

[Kyle Stanley]

May he rest in peace, I can't even fathom the impact PIL has had on Python
over the years.

On Fri, Dec 10, 2021 at 3:20 PM Guido van Rossum  wrote:

> A former core dev who works at Google just passed the news that Fredrik
> Lundh (also known as Effbot) has died.
>
> Fredrik was an early Python contributor (e.g. Elementtree and the 're'
> module) and his enthusiasm for the language and community were inspiring
> for all who encountered him or his work. He spent countless hours on
> comp.lang.python answering questions from newbies and advanced users alike.
>
> He also co-founded an early Python startup, Secret Labs AB, which among
> other software released an IDE named PythonWorks. Fredrik also created the
> Python Imaging Library (PIL) which is still THE way to interact with images
> in Python, now most often through its Pillow fork. His effbot.org site
> was a valuable resource for generations of Python users, especially its
> Tkinter documentation.
>
> Fredrik's private Facebook page contains the following message from
> November 25 by Ylva Larensson (translated from Swedish):
>
> """
>
> It is with such sorrow and pain that I write this. Fredrik has left us
> suddenly.
>
> """
>
> A core dev wrote: "I felt privileged to be able to study Fredrik's code
> and to read his writing. He was a huge asset to our community in the early
> days. I enjoyed his sense of humor as well. I'm sad that he passed away."
>
> We will miss him.
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> python-committers mailing list -- python-committ...@python.org
> To unsubscribe send an email to python-committers-le...@python.org
> https://mail.python.org/mailman3/lists/python-committers.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-committ...@python.org/message/36Q5QBILL3QIFIA3KHNGFBNJQKXKN7SD/
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YAIZPWDPVWD74ZTA3Z7G25VFQ2URAU2Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: peps, devguide, voters repositories: master branch renamed to main

[Kyle Stanley]

Thanks for the update! Although possibly a minor inconvenience to some, I
am certainly of the belief that our choice in language has a subtle, yet
powerful effect on our perception of the world around us, so although some
may see it as needless, I think small changes such as "master" to "main"
are necessary; assuming we want to evolve beyond archaic standards and
become more inclusive over time.

On Sat, Dec 4, 2021 at 4:07 AM Victor Stinner  wrote:

> Hi,
>
> The "master" branch of the following Python GitHub repositories have
> been renamed to "main":
>
> * devguide
> * peps
> * voters
>
> For the rationale of the rename, see:
> https://sfconservancy.org/news/2020/jun/23/gitbranchname/
>
> If you already have a Git checkout of one of these repositories, you
> can rename your local master branch with these commands (where "orign"
> is the github.com/python/... remote):
> ---
> git branch -m master main
> git fetch origin
> git branch -u origin/main main
> git remote set-head origin -a
> ---
>
> Tell me if you have any issue with the branch rename.
>
> Thanks Brett, Ee, Mariatta and the Steering Council who helped to
> rename these branches ;-)
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WUGZX7KW3B33LM7R7O2ON6NRQFDGCTSW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IUKOKO2YJNXPNVYLIJ5RFUPPOZBK6FU6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be careful about changing PEPs post-submission to the SC

[Kyle Stanley]

FWIW that seems like a reasonable approach, at least to me.

On Thu, Nov 18, 2021, 5:29 PM Guido van Rossum  wrote:

> I know PEP 646 was one of these. In our defense, we *did* notify the SC
> that there was a pending issue (
> https://github.com/python/steering-council/issues/59#issuecomment-951728233),
> although at the time we didn't anticipate it to become such a contentious
> discussion between the PEP authors. (Though, while contentious, it's still
> a minor edge case in the PEP, and I don't think it would affect the SC's
> position which way we eventually end up going.)
>
> I'm guessing that the recommended approach in such a case is just to close
> the SC issue and reopen it once the PEP is updated?
>
> On Thu, Nov 18, 2021 at 12:04 PM Brett Cannon  wrote:
>
>> This is a personal plea (i.e. not coming from the SC at all), but in the
>> last month we have had PEPs changed twice post-submission to the SC. That's
>> a big time sink as we take multiple meetings to discuss a PEP and having
>> things change underneath us causes us to have to re-evaluate our
>> discussions (and I know I pretty much start thinking about PEPs once they
>> are submitted, whether we are actively discussing them or not and I'm
>> probably not the only SC member who does this).
>>
>> I know no one did this maliciously or anything, but since it's happened
>> twice now I just want to ask people be cognizant of this. Please reach out
>> to the SC if you want to make a change so we can discuss whether we think
>> it will help/hurt the PEP, etc. and we are also not taken off-guard by
>> things shifting (assume we don't monitor the commits and PRs to the peps
>> repo, so unless you explicitly say, "hold on", we won't realize discussions
>> are ongoing in a PR or anything). If that means withdrawing your PEP for
>> consideration for a while that's totally fine and it won't hurt your
>> chances of acceptance once you're at a stable state with your PEP.
>>
>> Once again, this is a personal ask and no one is mad at anyone. I'm just
>> asking people be very clear in communicating with us when they want to make
>> a change to a PEP or they have suddenly have an open issue they are still
>> discussing after they open an issue in the steering-council repo for us to
>> review a PEP and need us to stop considering their PEP for a while.
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/ZLC3H2XLEYJLFV3TRQ2EWRKRGZZ7DRMC/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WJOTJ6V7XKRMDESEF3BD4DHOP4C3TRNK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/44UCT4MC2NGSHSWBHOBSB363G3RPA5IW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

[Kyle Stanley]

I think it would be fine to wait just one release, until 3.12. Makes no
substantial maintenance difference and maybe easier for users with more
advanced notice, especially for module removal.

I also wonder if maybe we should scale delay between dep -> removal based
on maintenance burden estimate, rather than 2 versions for all. Module
removal certainly takes more effort to adjust in code vs simple function
name change with 1:1 replacement.

-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F52UKISVFFGBNCL2AZ4XVX2KL35O6ZNH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Steering Council elections.

[Kyle Stanley]

On Mon, Nov 15, 2021 at 10:49 AM Thomas Wouters  wrote:

>
> Just a reminder that the nomination period for the next SC ends *today*
> (AoE), so if you're intending to nominate (yourself or someone else),
> please get those posts in. (No need for a long post before the deadline, it
> can be expanded later.) We currently have the 4 incumbents, and nobody
> else, so please consider who you would like to see on the SC.
>

Thanks for the reminder, Thomas. Although I have nobody to nominate or
consider myself to have the capacity and experience to self-nominate for
SC, it is great to see that the incumbents desire to have a wide pool of
applicants for the longevity of Python; rather than being on the SC
indefinitely.

Best Regards,
-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PHBQHDHQJMJPT26EPJMLANIPWQ6W2GFM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Kyle Stanley]

On Sat, Nov 13, 2021 at 5:04 PM  wrote:

>
>
> def 횑퓮햑풍표():
>
> try:
>
> 픥e헅핝횘︴ = "Hello"
>
> 함픬r퓵ᵈ﹎ = "World"
>
> ᵖ햗퐢혯퓽(f"{헵ｅ퓵픩º_}, {햜ₒ풓lⅆ︴}!")
>
> except 퓣핪ᵖｅ햤헿ᵣ햔횛 as ⅇ헑c:
>
> 풑rℹₙₜ("failed: {}".핗헼ʳᵐªｔ(ᵉ퐱퓬))
>
>
>
> if _︴ⁿ퓪푚핖__ == "__main__":
>
> 풉eℓˡ허()
>
>
>
>
>
> # snippet from unittest/util.py
>
> _퓟Ⅼ햠홲험ℋ풪Lᴰ푬핽﹏핷피헡 = 12
>
> def _픰ʰ퓸ʳ핥홚푛(픰, p푟픢fi햝핝횎푛, ｓᵤ푓헳헂푥헹ₑ횗):
>
> ˢ헸ｉ헽 = 퐥ｅ혯(햘) - ｐr횎햋퐢x헅ᵉ퓷 - 풔홪ffｉ혅헹홚ₙ
>
> if sｋi혱 > _퐏헟햠혊홴H핺Ｌ핯홀혙﹏L픈풩:
>
> 혴 = '%s[%d chars]%s' % (홨[:혱퐫핖푓핚ｘℓ풆핟], ₛ횔풊p, 퓼[퓁풆햓(
> 횜) - 홨횞풇fix홡ᵉ혯:])
>
> return ₛ
>

0_o color me impressed, I did not think that would be legal syntax. Would
be interesting to include in a textbook, if for nothing else other than to
academically demonstrate that it is possible, as I suspect many are not
aware.

-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4ZA2KK46JHENKSPB52RMXBAQT7CP3Q6A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pre-PEP: Unicode Security Considerations for Python

[Kyle Stanley]

I'd suggest both: briefer, easier to read write up for average user in
docs, more details/semantics in informational PEP. Thanks for working on
this, Petr!

On Tue, Nov 2, 2021 at 2:07 PM David Mertz, Ph.D. 
wrote:

> This is an amazing document, Petr. Really great work!
>
> I think I agree with Marc-André that putting it in the actual Python
> documentation would give it more visibility than in a PEP.
>
> On Tue, Nov 2, 2021, 1:06 PM Marc-Andre Lemburg  wrote:
>
>> On 01.11.2021 13:17, Petr Viktorin wrote:
>> >> PEP: 
>> >> Title: Unicode Security Considerations for Python
>> >> Author: Petr Viktorin 
>> >> Status: Active
>> >> Type: Informational
>> >> Content-Type: text/x-rst
>> >> Created: 01-Nov-2021
>> >> Post-History:
>>
>> Thanks for writing this up. I'm not sure whether a PEP is the right place
>> for such documentation, though. Wouldn't it be more visible in the
>> standard
>> Python documentation ?
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Experts (#1, Nov 02 2021)
>> >>> Python Projects, Coaching and Support ...https://www.egenix.com/
>> >>> Python Product Development ...https://consulting.egenix.com/
>> 
>>
>> ::: We implement business ideas - efficiently in both time and costs :::
>>
>>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>Registered at Amtsgericht Duesseldorf: HRB 46611
>>https://www.egenix.com/company/contact/
>>  https://www.malemburg.com/
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/FSFG2B3LCWU5PQWX3WRIOJGNV2JFW4AU/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/6PHPDZRCYNA44NHSHXPBL7QMWXMHXWGU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6OET4CKEZIA34PAXIJR7BUDKT2DPX2DG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Bug fixed in the bugs.python.org OAuth-based authentication: user logged as the wrong account

[Kyle Stanley]

Thanks for the fix! This could have caused some serious issues, so glad we
were able to address it ahead of time.

On Mon, Sep 13, 2021 at 5:06 AM Victor Stinner  wrote:

> Hi,
>
> A bug has been identified and *fixed* in the OAuth-based
> authentication code used on the Python bug tracker bugs.python.org
> (BPO) to log in with GitHub, Launchpad or Google. Under some
> conditions, it was possible to be logged as another person account. We
> are only aware of a single user affected by the issue. We are not
> aware of any account takeover.
>
> All bugs at bugs.python.org are public: being logged as the wrong
> account cannot give access to private bugs. The main risk is if an
> attacker could be logged as an administrator (the "Coordinator" role)
> which allows to change the bug tracker configuration and to change
> accounts (add/remove roles, see/change the email address, etc.). We
> are not aware of any abuse.
>
> All OAuth accounts have been removed in the database to fully fix the
> issue. Users using OAuth-based authentication must associate again
> (once) their GitHub, Launchpad or Google account with their BPO
> account.
>
> A BPO account contains the following information: Name, Login Name,
> GitHub Name, Organisation, Timezone, Homepage, Contributor Form
> Received, Is Committer, E-mail address, Alternate E-mail addresses.
> All fields but Name and Timezone are hidden to other accounts, only
> coordinators can see all fields of other accounts. You can check in
> the "Your Details" page for the your account change log.
>
> Thanks Ammar Askar, Berker Peksağ and Ee Durbin who fixed the bug!
>
> Source code of bugs.python.org (Roundup fork):
> https://github.com/psf/bpo-tracker-cpython
>
> The OAuth-based authentication is an extension written for
> bugs.python.org. The bug report and its fix:
>
> * https://github.com/python/bugs.python.org/issues/64
> *
> https://github.com/psf/bpo-tracker-cpython/commit/0a32e072aafca20c0bf51cf16fb6a7328cdd720a
>
> Report issues with bugs.python.org:
> https://github.com/python/bugs.python.org/issues
>
> To report sensitive issues, write to: secur...@python.org
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/CIXIB6EMN7HOPMXFJI7EBK7V7OPK4E7H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OH7V7GJ6GTQJM3OBIXZ72IZXA4KSLVVH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Kyle Stanley]

I can agree with the general premise of what Antoine is saying, but to me
even as a non-participant, the following quote from the thread Brett linked
seems a clear CoC violation:

I repeat, even the worst AI will understand from the context what I meant.
> But let me do a very rude example:
>
> What if I said to Steven "I pretend immediate suck my $%#$"? Do you think
> you and the others will not understand the sense? :D
>

This at least indicates a lengthy timeout is needed, imo.

On Mon, Aug 16, 2021 at 1:01 PM Antoine Pitrou  wrote:

> On Mon, 16 Aug 2021 09:47:13 -0700
> Brett Cannon  wrote:
> >
> https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/
> >
> > I can't be objective with Marco as I believe we have recorded issues with
> > him previously (as with Steven if you take Marco's initial side with
> this).
> >
> > The thing that pushed me over the edge to report this was
> >
> https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/
> > .
> >
>
> I think a private appeal to the two or three participants to remain
> cool, nice and polite would be welcome here.  Also an appeal to
> think twice before posting... (is what you're going to say really going
> to benefit the community?)
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OUKC6WY5VCAD2ZIKQZQUHOP5LUTSDMF7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L5SEEI5DLPU2UMGERSKTQCMVCG4TAAQI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Bug report

[Kyle Stanley]

On Wed, Jul 21, 2021 at 12:19 PM Nguyen Do Minh Duc <
gd.minhduc1...@gmail.com> wrote:

> Hi,
>
> When I find what's new in 3.10 beta in
> https://docs.python.org/whatsnew/3.10.html
> It redirected me to https://docs.python.org/3/whatsnew/3.10.html which
> shows 404 error not found nginx. Can you fix this?
>

Thanks for the report! I don't believe the code and/or config for the web
server hosting docs.python.org is a public repo (may be mistaken), so I'll
CC the director of infrastructure, Ee W. Durbin. I suspect they will know
who to forward this to (or be able to fix it).

Best Regards,
-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WAJM6PFMX2DXOGPEC6I2QATAUT2GNJXC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Is the Python review process flawed?

[Kyle Stanley]

On Thu, Jul 1, 2021 at 10:49 AM Victor Stinner  wrote:

> What happens usually is that some modules have no maintainer. Once you
> merge a single change in a module, boom! You are now the new
> maintainer for your entire life time! More and more people will ask
> you to look at their "very important" bug blocking their production
> and their very specific use case. You will get more and more reviews
> request: "since you merged a single change 12 months ago, I'm sure
> that you will love to review my precius bugfix! Come on, it's
> trivial!". Now you realize that you don't know the design of the
> module. You don't know its history. You know nothing, and the entire
> world now have very high expectation, since you merged an obvious and
> trivial change.
>
“The ancient Oracle said that I was the wisest of all the Greeks. It is
because I alone, of all the Greeks, know that I know nothing.” ~  Socrates

The same could largely be applied to maintaining open source (especially a
codebase as large as CPython). The truth is that at a fundamental level,
we're all (any software developer) just making educated guesses as to
whether or not patches are suitable for merging, or whether or not a bug
fix will actually work. Tests make us a little more certain, but it is
still a guess. So the problem is really just the unrealistic expectation
that there exists people who are able to make a decision with absolute 100%
certainty that no regressions or future unexpected issues will occur as a
result of the change.

Also, for fun, here's a pythonized version of the quote:
“[Guido] said that I was the wisest of all the [pythonistas]. It is because
I alone, of all the [pythonistas], know that I know nothing.”
:)

With loving-kindness,
-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SGDPAXGBI2627HFBK7NV5GQWDXDSRR4W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why list.sort() uses mergesort and not timsort?

[Kyle Stanley]

On Sun, Jun 6, 2021 at 7:09 PM Dan Stromberg  wrote:

> I've got a comparison of sort algorithms in both Cython and Pure Python
> (your choice) at:
> https://stromberg.dnsalias.org/~strombrg/sort-comparison/
> ...including a version of timsort that is in Cython or Pure Python.
>

Thanks for sharing the graphs. I found the performance of radix sort in
particular to be interesting to see mapped out visually, and have never
heard of "shellsort" prior to now. :)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BVHSMLAOJKVDXJO37LF5WY7OEM6D5YWX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 661: Sentinel Values

[Kyle Stanley]

 As someone who's had to make use of the pattern `_sentinel = object()` a
few times within stdlib code, I'd like to give a strong +1 for the proposal
to add a new `sentinel()` function. This is much more intuitive, easier to
look up, etc. From my experience, it's a common enough pattern to be well
worth the addition.

Thanks for the proposal, Tal. :)

On Sun, Jun 6, 2021 at 4:14 AM Tal Einat  wrote:

> Hi,
>
> I have prepared a PEP proposing adding a stdlib function for defining
> sentinel values. You can find the PEP here:
>
> https://www.python.org/dev/peps/pep-0661/
>
> The discussion is happening in the discourse server:
>
> https://discuss.python.org/t/pep-661-sentinel-values/9126
>
> To avoid splitting the discussion, *please redirect your comments there*
> instead of replying to this thread.
>
> Have a nice week,
> - Tal Einat
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3QNLVLSMVSQ5MGLXRIQ5QM4BA5OJCVEN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UM7EPUUSQPJAEU6QCA3FUOH3ULNUSXGV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Farewell, for now :)

[Kyle Stanley]

Hi all,

Last week, I started a new thread on discuss.python.org
 about my
intention to take a further extended break from open source to continue my
mental health healing process. Just wanted to announce it in the other
channels as well since I know that not everyone has the bandwidth to keep
up with more than just the MLs.

In the thread, I discussed my intention to pursue the path of becoming a
Buddhist monk for some time, and recently detailed my adventures at a local
Thai temple. Check it out if you are interested. :)

With Loving Regards,
-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5KN37I5PSMLSBEER3NWWVQCP7DBYPD3Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] The importance of mental health

[Kyle Stanley]

Hey all,

In these last few months, I have been in the process of healing from some
pretty heavy past trauma. And now that I am on the road to recovery, I want
to share my journey with the Python community in hopes that it may reach
those that are struggling with their own mental health battles, as many of
us are during these dark and difficult times.

Trigger warning that it includes a decent amount of highly personal
content, so only check it out if you are okay with that:
https://discuss.python.org/t/break-from-open-source/6372/7?u=aeros.

To anyone that would limit my employment opportunities as a result of
having had these struggles, *that's perfectly okay*. I kept the post in the
private section because I was originally in fear of discriminate. However,
I have reached an important conclusion: *I would not want to work for your
company if you discriminate against people who have overcome past struggles*
.

-- 
--Kyle R. Stanley, Python Core Developer (what is a core dev?
)
*Pronouns: they/them **(why is my pronoun here?*

)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3H6JIULWOTFYMJLWOBFO7PM4RAOEL74U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please explain how to migrate when a function is removed, thanks ;-)

[Kyle Stanley]

On Jan 20, 2021 at 9:51 PM Chros Jerdonek  wrote:

> Is there / would it make sense to have a section analogous to "Porting to
> Python X" that covers "Make All DeprecationWarnings Go Away in X"? If we
> had such a section, the "Porting to" section could be constructed by
> copying the relevant bits from that section in the previous release.


+1 from me to include this section in 3.10 and future whatsnew.

On Thu, Jan 21, 2021 at 2:17 PM Mariatta  wrote:

> I agree that when we land a feature that introduces incompatible change
> like this, as part of the PR it should also include updating the
> documentation on how to migrate.
> I would think that the feature should not be merged unless the doc has
> been updated too.
>
> Perhaps we should include this explicitly in devguide, as one of the
> things to consider when reviewing pull requests.
> Do we have such a guide/doc yet, on how to review CPython pull requests?
>

This seems like it would be a good location to include that information:
https://devguide.python.org/pullrequest/#how-to-review-a-pull-request

Maybe also a brief mention in the checklist for
https://devguide.python.org/committing/#is-the-pr-ready-to-be-accepted.
Step 9 currently has: "Was “What’s New” updated (as appropriate)?", which
could have a sub-point to mention "Incompatible changes should include
migration information in their respective "What's New".".
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E7NYJ5DUQM46GXKCQTUWRXBJUQ5DOAHR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please explain how to migrate when a function is removed, thanks ;-)

[Kyle Stanley]

Thanks for bringing attention to this, Victor, and to Ken Jin (GH:
Fidget-Spinner) for the PR. I've just completed reviewing and merging the
PR, so hopefully anyone affected will now have a more clear idea of how to
migrate their asyncio code to 3.10. Having the porting method explicitly
documented certainly helps to smooth the version transition process and
reduce headaches. :-)

Going forward, I'll try to make more of an active effort to ensure any
potentially incompatible changes I'm involved with include a clear method
of porting documented in their respective whatsnew. It can be easy to
forget at times that a seemingly minor fix which is intuitively clear to
the authors of a change may not be as clear to those not involved with it,
regardless of how difficult the fix actually is.

On Tue, Jan 19, 2021 at 12:03 PM Victor Stinner  wrote:

> A PR was proposed to document the removal of the loop parameter:
> https://github.com/python/cpython/pull/24256
>
> Victor
>
> On Tue, Jan 19, 2021 at 1:34 PM Victor Stinner 
> wrote:
> >
> > Hi,
> >
> > We are working on upgrading Python from 3.9 to 3.10 in Fedora and we
> > are facing many Python 3.10 incompatible changes. Most changes were
> > planned with a deprecation period, but, as usual, projects are not
> > tested with DeprecationWarning treated as errors, and so tons of
> > packages now fail to build.
> >
> > I'm not here to talk about DeprecationWarning, but how we communicate
> > on incompatible changes made on purpose. For example, What's New in
> > Python 3.8 announces: "In asyncio, the explicit passing of a loop
> > argument has been deprecated and will be removed in version 3.10". As
> > expected, the parameter was removed in Python 3.10 (bpo-42392), but I
> > cannot see anything in What's New in Python 3.10. The problem is that
> > I don't know how to fix projects broken by this change. I'm not
> > complaining about this specific change, but more generally.
> >
> > I strongly suggest to well document incompatible changes. The bare
> > minimum is to mention them in "Porting to Python 3.10" section:
> > https://docs.python.org/dev/whatsnew/3.10.html#porting-to-python-3-10
> >
> > A link to the bpo sometimes helps to understand how to port code. But
> > I would really appreciate if authors of incompatible changes would
> > explain how to add Python 3.10 support to existing projects, without
> > losing support for older Python versions. Not just "this function is
> > now removed, good luck!" :-)
> >
> > I didn't touch asyncio for at least 1 year, so I don't know what
> > happens if I remove a loop argument. Does an application remain
> > compatible with Python 3.6 without passing loop?
> >
> > I know that I made multiple incompatible changes myself and I'm sure
> > that the documentation should probably be enhanced, but I also expect
> > others to help on that! ;-)
> >
> > Please think about people who have to port 4000 Python projects to
> Python 3.10!
> >
> > ---
> >
> > The best would be to ship a tool which adds Python 3.10 support to
> > existing projects without losing support with Python 3.6. Maybe
> > something like https://github.com/asottile/pyupgrade could be used for
> > that? pyupgrade looks more specific to the Python syntax, than the
> > usage of the stdlib.
> >
> > I wrote such tool to add Python 3.10 support to C extensions without
> > losing support with Python 2.7. It relies on a header file
> > (pythoncapi_compat.h) which provides new C API functions on old Python
> > versions.
> > https://github.com/pythoncapi/pythoncapi_compat
> >
> > For example, it replaces "obj->ob_refcnt = refcnt;" with
> > "Py_SET_REFCNT(obj, refcnt);" and provides Py_SET_REFCNT() to Python
> > 3.8 and older (function added to Python 3.9)
> >
> > Victor
> > --
> > Night gathers, and now my watch begins. It shall not end until my death.
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/CVLDV7VIVENV6HMQ4PRAN3VRFU26CMJI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RGSEPSSPDIVKWYKACONKRUFBEXQ5YKGC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 642 v2: Explicit constraint patterns *without* question marks in the syntax

[Kyle Stanley]

On Sun, Nov 15, 2020 at 1:56 AM Chris Angelico  wrote:

> On Sun, Nov 15, 2020 at 4:28 PM Kyle Stanley  wrote:
> >
> > FWIW, I'd like to add my +1 to usage of "as" for spelling class capture
> patterns. This is by far the clearest and easiest to read form I've seen
> thus far, and I suspect that it would be the easiest to explain to users
> already familiar with usage of "as" from other areas in Python. A new
> feature being as distinguishable as possible and easy to explain to
> existing users is very important in my book, and based on the responses, I
> think that the current "=" form used in PEP 634 for matching class patterns
> would be substantially more difficult for users to mentally parse and
> understand compared to "as".
> >
> > It's also worth considering new Python users that might have general OO
> experience but not with robust pattern matching (e.g. Java-heavy
> backgrounds). I could definitely see "case Point(x=a, y=b):" being confused
> for instantiation, whereas usage of "as" makes it more clear that something
> else is happening (hopefully leading them to search around for more info
> about Python pattern matching).
> >
>
> case Point(x=as a, y=as b):
>
> That doesn't read well to me.
>
> Or is there some other spelling of 'as' that makes better sense to you?
>

The post from Nick that I was primarily replying to used the following
spelling: "case Point(x as a, y as b):".
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JBSWERRHKBYAB7OW5HS3E4FGVTXDTJUP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 642 v2: Explicit constraint patterns *without* question marks in the syntax

[Kyle Stanley]

On Sat, Nov 14, 2020 at 7:54 AM Nick Coghlan  wrote:

> On Sat, 14 Nov 2020 at 09:51, Greg Ewing 
> wrote:
> >
> > On 14/11/20 7:45 am, Brandt Bucher wrote:
> > > with (using your own syntactic flavor):
> > > ```
> > > case >first, *>middle, >last:
> > >  rebuilt = first, *middle, last
> > > case {"key": >value, **>rest}:
> > >  rebuilt = {"key": value, **rest}
> > > case Point(x=>a, y=>b):
> > >  rebuilt = Point(x=a, y=b)
> >
> > I think this is a case where syntax matters. To my eyes this
> > looks far less confusing:
> >
> > case ?first, *?middle, ?last:
> >   rebuilt = first, *middle, last
> > case {"key": ?value, **?rest}:
> >   rebuilt = {"key": value, **rest}
> > case Point(x=?a, y=?b):
> >   rebuilt = Point(x=a, y=b)
>
> Based on the discussion in this thread, v3 of PEP 642 is going to
> propose spelling these as:
>
> case first, *middle, last:
>  rebuilt = first, *middle, last
> case {"key" as value, **rest}:
>  rebuilt = {"key": value, **rest}
> case Point(x as a, y as b):
>  rebuilt = Point(x=a, y=b)
>
> I'm in agreement with the PEP 634 authors that we want to keep
> sequence pattern matching consistent with iterable unpacking as far as
> name binding is concerned. Mapping patterns and class patterns are
> both completely new though, and I think the readability problem there
> is quite similar to the one that existed with walrus patterns in PEP
> 622, and thus amenable to a similar solution (i.e. spell the capture
> patterns for those cases with `as`, not `:` or `=`).
>

FWIW, I'd like to add my +1 to usage of "as" for spelling class capture
patterns. This is by far the clearest and easiest to read form I've seen
thus far, and I suspect that it would be the easiest to explain to users
already familiar with usage of "as" from other areas in Python. A new
feature being as distinguishable as possible and easy to explain to
existing users is very important in my book, and based on the responses, I
think that the current "=" form used in PEP 634 for matching class patterns
would be substantially more difficult for users to mentally parse and
understand compared to "as".

It's also worth considering new Python users that might have general OO
experience but not with robust pattern matching (e.g. Java-heavy
backgrounds). I could definitely see "case Point(x=a, y=b):" being confused
for instantiation, whereas usage of "as" makes it more clear that something
else is happening (hopefully leading them to search around for more info
about Python pattern matching).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VK6YAMK5NSOZRNRDHLHL7UH67LGVL3HU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please do not remove random bits of information from the tutorial

[Kyle Stanley]

 Rather than trying to specifically transform the existing tutorial into a
guide exclusively aimed at beginners, I think that we should use the
guideline of: "Is this useful information in 95% of real-world use cases or
does it have a strong niche purpose that will be useful at *some *point for
significant number of Python users?" If not, my opinion is that it should
be removed and replaced with a footnote link for more information (if the
existing documentation is too technical, we can have it linked to a
separate HOWTO which explains it in a way that's easier to parse).

On Sat, Nov 7, 2020 at 9:16 PM Inada Naoki  wrote:

> I think `from exc` syntax is not new Python users should know.
> Documenting implicit chaining is enough for 99% use cases, and `from
> None` covers 0.99% of the rest.
> So I considered removing explicit chaining (e.g. `from exc`) from the
> section too.
> But I kept it, because it's a "syntax showcase" even though it will
> give more noise to beginners.
>
> And deleting `__cause__` is not solery for "didactic" reason, nor
> "loosing something precious."
> As written in the b.p.o. issue, mention only `__cause__` "lose some
> precious".
> We need to describe `__context__` and `__suppress_context__` too.
> But all of them are documented in library/exceptions.html.
> Removing `__cause__` and adding a link to library/exceptions.html
> makes sense more than documenting all.
>

So, I think that in a case where as Inada mentioned above, it's not going
to be needed for 99% of use cases, my opinion is that the extra information
is just consuming valuable real estate in the tutorial that adds a higher
burden to readers trying to learn Python for little gain. I'm less
concerned about removing parts that are "too difficult" and more concerned
with removing parts that are never realistically going to be useful for the
vast majority of users (outside of something like implementing the
internals of some ultra-niche library).

__cause__ is a great example of this, but I would be hesitant to remove
explicit chaining; that would require much more consideration because it
has a significantly better chance of being relevant for users or at least
appearing in code at some point without specifically trying to find it. So
I agree with keeping the explicit exception chaining part.

There shouldn't be a concern that we'll be "removing random bits of
information from the tutorial", any change to the tutorial will be made
intentionally and with much consideration. Also, even as a core dev that
has spent a decent amount of time reviewing CPython documentation PRs, I
would personally wait until having 3+ opinions prior to merging any
removals of information or other substantial changes in the tutorial (with
at least one other core dev, preferably two), and encourage others to do
the same since the tutorial is a very fundamental part of the docs.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2XKNMFRSK2YUDFDCQFX7XLIHUFQOLTAH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please do not remove random bits of information from the tutorial

[Kyle Stanley]

On Sun, Nov 8, 2020 at 8:17 PM Inada Naoki  wrote:

> OK. Since checking all mails in the long thread is tedious job, I will
> pick some up and leave a comment in the b.p.o.
>

Personally, I think that just linking to the python-dev thread in bpo
(and/or PR) is adequate for most cases, especially if there was a fairly
strong consensus pointing in one direction.

(More detail can be helpful of course, but if one can only spare enough
time to just place a link on the issue, it does quite a lot by itself IMO.)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AF2QAEPVWPSWS6ANXXUHUUMFKR5THPC3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Who is target reader of tutorial?

[Kyle Stanley]

I can't speak for all of the members of the upcoming documentation WG, but
as someone that will be on it (based on our discussions at the recent core
dev sprint), my personal vote would be for keeping it as a comprehensive
guide for beginners of Python. Detailed enough that it covers the
fundamentals, but skips out on more advanced topics and info that is likely
going to be less applicable to most new users.

I would consider __cause__ and __context__ to fall under the category of
being "useful to know, but far from essential for beginners". You can have
a decent understanding of most python programs without ever knowing about
those two dunders.  So I would definitely be in favor of removing the
mention of __cause__ and not adding __context__, in my opinion it adds
extra unneeded complexity.

That being said, I'm not opposed to something like having something like a
footnote at the bottom of the section that links to a more advanced topic
(maybe an exception how-to?). It should be at the bottom of the section so
it doesn't add extra cognitive overhead for those trying to grasp the
basics, but allows for those who are interested to get a more in-depth
understanding if they want to. I would suggest redirecting the contributor
who proposed those changes to working on something like an exception
how-to; assuming that's something they're interested in.

On Thu, Nov 5, 2020 at 2:07 PM Brett Cannon  wrote:

> A documentation WG is going to be formed which will be in a better
> position to answer this, so until that WG is started I think we should keep
> the tutorial aimed towards beginners.
>
> On Thu, Nov 5, 2020 at 1:13 AM Inada Naoki  wrote:
>
>> Hi, all.
>>
>> Since "How To" guide is not organized well, it is very tempting to
>> write all details in tutorial.
>> I have seen may pull requests which "improve" tutorial by describe
>> more details and make
>> the tutorial more perfect.
>>
>> But adding more and more information into tutorial makes it slower to
>> learn Python by reading tutorial.
>> 10+ years ago, Python is far less popular in Japan and reading
>> tutorial is the best way to learn Python to me.
>>
>> But now Python is popular and there are many free/paid good books and
>> tutorials on the Web.
>> Some of them would be more new user friendly than official tutorial.
>> Then, should official Python tutorial become detailed guide to Python?
>> Or should we keep new user learning Python as targeted reader?
>>
>> There is ongoing issue for example: https://bugs.python.org/issue42179
>>
>> Chaining exception was added in tutorial.  Current tutorial mention to
>> `__cause__` attribute.
>> https://docs.python.org/3/tutorial/errors.html#exception-chaining
>>
>> bpo-42179 proposes to add mention to `__context__` to make the
>> tutorial more accurate about implicit chaining.
>> And https://github.com/python/cpython/pull/23160 is the pull request
>> to mention `__context__`.
>>
>> On the other hand, I want to remove confusion by removing mention to
>> `__cause__`.
>> Because I don't think `__context__` and `__cause__` is important for new
>> users.
>> See https://github.com/python/cpython/pull/23162 for my proposal.
>>
>> Regards,
>>
>> --
>> Inada Naoki  
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/MXMEFFYB6JXAKSS36SZ7DX4ASP6APWFP/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IWW2YBLJK4T3OWSKDUDVDVXPWDGIFWTC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FWQWF6BNVILXLYGZ7VVFBWP7OV23URSD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

[Kyle Stanley]

+1 to remove support for Solaris going forward. 4 years is plenty of time
to wait for someone to volunteer to maintain it, IMO. So my preference
would be for option 3 to remove it now, but I wouldn't be opposed to option
2 either w/ deprecating support and waiting a couple versions to remove it.
I'm only opposed to option 1, since it seems very likely that it will just
continue to stagnate at this point.

On Thu, Oct 29, 2020 at 5:49 PM Victor Stinner  wrote:

> Hi,
>
> I propose to drop the Solaris support in Python to reduce the Python
> maintenance burden:
>
>https://bugs.python.org/issue42173
>
> I wrote a draft PR to show how much code could be removed (around 700
> lines in 65 files):
>
>https://github.com/python/cpython/pull/23002/files
>
> In 2016, I asked if we still wanted to maintain the Solaris support in
> Python, because Solaris buildbots were failing for longer than 6
> months and nobody was able to fix them. It was requested to find a
> core developer volunteer to fix Solaris issues and to set up a Solaris
> buildbot.
>
>
> https://mail.python.org/archives/list/python-dev@python.org/thread/NOT2RORSNX72ZLUHK2UUGBD4GTPNKBUS/#NOT2RORSNX72ZLUHK2UUGBD4GTPNKBUS
>
> Four years later, nothing has happened. Moreover, in 2018, Oracle laid
> off the Solaris development engineering staff. There are around 25
> open Python bugs specific to Solaris.
>
> I see 3 options:
>
> * Current best effort support (no change): changes only happen if a
> core dev volunteers to review and merge a change written by a
> contributor.
>
> * Schedule the removal in 2 Python releases (Python 3.12) and start to
> announce that Solaris support is going to be removed
>
> * Remove the Solaris code right now (my proposition): Solaris code
> will have to be maintained outside the official Python code base, as
> "downstream patches"
>
>
> Solaris has a few specific features visible at the Python level:
> select.devpoll, os.stat().st_fstype and stat.S_ISDOOR().
>
> While it's unclear to me if Oracle still actively maintains Solaris
> (latest release in 2018, no major update since 2018), Illumos and
> OpenSolaris (variants or "forks") still seem to be active.
>
> In 2019, a Solaris blog post explains that Solaris 11.4 still uses
> Python 2.7 but plans to migrate to Python 3, and Python 3.4 is also
> available. These two Python versions are no longer supported.
>
> https://blogs.oracle.com/solaris/future-of-python-on-solaris
>
> The question is if the Python project has to maintain the Solaris
> specific code or if this code should now be maintained outside Python.
>
> What do you think? Should we wait 5 more years? Should we expect a
> company will offer to maintain the Solaris support? Is there a
> motivated core developer to fix Solaris issue? As I wrote, nothing has
> happened in the last 4 years...
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VDD7NMEDFXMOP4S74GEYJUHJRJPK2UR3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N3CQPEPNENRC6L7NIALN3JGA3O33UDD3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Core Dev Sprint: Collecting Discord usernames

[Kyle Stanley]

On Fri, Oct 2, 2020 at 2:07 AM Kyle Stanley  wrote:

> Prior to joining Python Discord, I recommend checking out the Discord
> setup guide that I recently finished:
> https://python-core-sprint-2020.readthedocs.io/communication.html#discord-setup-guide.
> The part on the privacy settings is particularly important for anyone with
> a public following to avoid receiving mass DMs and friend requests.
>
> Alternatively, you can join directly at the following perma-link:
> https://discord.gg/Q87A9Y9.
>

Minor clarification: since there was mention that the "alternatively" part
was unclear, I was just referring to joining the server directly for those
that didn't have time available to read over the guide prior to joining
(the guide includes a section on joining the server via the link).

Also, since we will be granting the role to participants en masse via
script, it will be granted after the majority of participants have
responded. As of writing this, I've received ~10/41 responses.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZCQMRVDVWOB36RKGTJMO4SRJYFMUC46B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Core Dev Sprint: Collecting Discord usernames

[Kyle Stanley]

Hey all,

The upcoming 2020 Python Core Development Sprint is right around the
corner, and we're currently towards the end of finalizing platform setup.

For using Discord during the sprint on the public Python Discord server (as
voted on in
https://discuss.python.org/t/2020-core-dev-sprint-poll-discord-or-zulip/5016/6),
we are collecting Discord usernames ahead of time and requesting that all
participants join Python Discord well before the start of the sprint (Oct.
19th). This will allow us to grant all participants a role for joining the
sprint channels (in order to be given the role, participants must be on the
server first).

To send your Discord username, simply reply to this email (without CCing
the list to keep it private) with it included. It is in the format of
"name#0123", and can be seen in the lower left corner of the Discord client
after creating an account (or in "User Settings" > "My Account"). I'll
verify replies against the email address used in the signup form, so please
use the same one if possible.

Prior to joining Python Discord, I recommend checking out the Discord setup
guide that I recently finished:
https://python-core-sprint-2020.readthedocs.io/communication.html#discord-setup-guide.
The part on the privacy settings is particularly important for anyone with
a public following to avoid receiving mass DMs and friend requests.

Alternatively, you can join directly at the following perma-link:
https://discord.gg/Q87A9Y9.

As a reminder, potential participants include Python core developers,
triagers, and those in a core dev mentorship. If you haven't already signed
up and are interested in attending, please do so at
https://forms.gle/fhzJdpRHR4GtSRCk9.

Regards,
Kyle Stanley
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H3BFBA2HH5SQ222DL27YFI3ABKAONAKI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Doc tests failing for many PRs on GitHub

[Kyle Stanley]

Thanks for reverting the setuptools version Ned, and to Victor for opening
a PR to make the fix for the latest version. I'm always amazed by the
efforts made and quick responses to keep things running smoothly. :-)

On Tue, Sep 1, 2020 at 5:56 AM Ned Deily  wrote:

> On Sep 1, 2020, at 05:47, Ned Deily  wrote:
> > I've opened https://bugs.python.org/issue41685 and pushed a temporary
> change to the Docs build venv to pin the setuptools version to a previous
> version.  I've also restarted the failing CI test for Mark's PRs and a
> couple of others.
>
> But, of course, just restarting the failed CI test isn't going to work
> right away since the PRs don't include the new setuptools pin.  So either
> cause the PRs to be updated against the current HEAD of master by pushing
> new commits to them or wait for a fixed version of setuptools and then
> cause the CI to rerun.
>
> I'll let the affected PR authors take care of that :)
>
>
> > On Sep 1, 2020, at 04:35, Victor Stinner  wrote:
> >> The documentation job of our pre-commit CI fails because of a change
> >> in the newly released setuptools 50.0: it comes from the new
> >> _distutil_hack module of setuptools.
> >>
> >> I proposed https://github.com/pypa/setuptools/pull/2361 to fix Python
> >> 3.10 support in the _distutils_hack module.
> >>
> >> Sadly, in the meanwhile, it's no longer possible to merge any PR in
> >> Python since the documentation job is mandatory. Either we wait until
> >> setuptools is fixed, or we make the documentation job non-voting.
> >>
> >> I propose to wait until setuptools is fixed and use this time to
> >> review a bunch of pending pull requests :-)
> >>
> >> Victor
> >>
> >> Le lun. 31 août 2020 à 19:30, Ammar Askar  a
> écrit :
> >>>
> >>> Filed https://github.com/pypa/setuptools/issues/2362 against
> >>> setuptools, not sure what we should do locally to fix this in the
> >>> meantime.
> >>>
> >>> On Mon, Aug 31, 2020 at 1:11 PM Ammar Askar 
> wrote:
> 
>  Karthik, your analysis is correct. You just have to import importlib
>  first to get the error:
> 
> >>> import importlib
> >>> import importlib.util
> >>> importlib.util.abc.Loader
>  Traceback (most recent call last):
>  File "", line 1, in 
>  AttributeError: module 'importlib.util' has no attribute 'abc'
> 
>  On Mon, Aug 31, 2020 at 12:09 PM Karthikeyan 
> wrote:
> >
> > The last successful build is
> https://travis-ci.com/github/python/cpython/jobs/379161961
> > Recent failure build is
> https://travis-ci.com/github/python/cpython/jobs/379329436
> >
> > I can see setuptools being upgraded from 49.6.0 to 50.0.0 as a
> difference. I can see the below command to print sphinx-build version error
> out in the makefile but unfortunately the error is being redirected to
> /dev/null in the makefile and I needed to run it explicitly to see the
> output. There is a recent change that moves Loader to _abc and imports
> Loader from _abc inside abc is done with below commit. The same code is
> present in 49.6.0 too at
> https://github.com/pypa/setuptools/blob/04e3df22df840c6bb244e9b27bc56750c44b7c85/_distutils_hack/__init__.py#L79
> . So I am not sure why this causes error.
> >
> > Pinning the dependency of setuptools to 49.6.0 in Doc/makefile runs
> the tests fine without error. Can you please try pinning it and see if it
> helps?
> >
> > I am not sure if the below commit is the cause that is picked up in
> setuptools but someone can correct me if I am wrong about the importlib
> machinery error.
> >
> > commit 9e09849d20987c131b28bcdd252e53440d4cd1b3
> > Author: Victor Stinner 
> > Date:   Wed Jun 17 23:15:59 2020 +0200
> >
> >   bpo-41006: importlib.util no longer imports typing (GH-20938)
> >
> >   Create importlib._abc submodule to avoid importing typing when
> >   importlib.util is imported. Move Loader ABC into importlib._abc.
> >
> > ./python
> > Python 3.10.0a0 (heads/master:75c80b0bda, Aug 30 2020, 13:53:05)
> > [GCC 7.5.0] on linux
> > Type "help", "copyright", "credits" or "license" for more
> information.
>  import importlib.util
>  importlib.util.abc.Loader
> > Traceback (most recent call last):
> > File "", line 1, in 
> > AttributeError: module 'importlib.util' has no attribute 'abc'
> 
> > ➜  cpython git:(master) ✗ git checkout
> 9e09849d20987c131b28bcdd252e53440d4cd1b3~1 Lib/importlib/util.py
> > ➜  cpython git:(master) ✗ ./python
> > Python 3.10.0a0 (heads/master:75c80b0bda, Aug 30 2020, 13:53:05)
> > [GCC 7.5.0] on linux
> > Type "help", "copyright", "credits" or "license" for more
> information.
>  import importlib.util
>  importlib.util.abc.Loader
> > 
> >
> > xvfb-run make -C Doc/ PYTHON=../python SPHINXOPTS="-q -W
> --keep-going -j4" doctest suspicious html
> > make: Entering directory '/root/cpython/Doc'
> > make[1]: Entering 

[Python-Dev] RSVP: 2020 Python Core Dev Sprint

[Kyle Stanley]

Hey all,

We just recently finished working on the signup form for the upcoming 2020
Python Core Dev Sprint on Oct. 19th - 23rd. In addition to getting an
general idea of (virtual) attendance, this is being used to assess the
topics covered, optimal start/end times for each day, and preferred social
activities.

Anyone who is a CPython core developer, engaged in a core dev mentorship,
or a member of the Python triage team is welcome to attend!

Sign up to attend here: https://forms.gle/84NkyYt5g616fs1r5

We will be accepting late sign ups for this event since capacity is not a
significant limitation, but for the purposes of scheduling the best
possible times for everyone, it is requested that participants do so at
their earliest convenience.

Regards,
Kyle Stanley
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C24BIJVTG53G4X25K5QLPP2WPKDCI2XA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pay for PR review and merging for VxWorks RTOS

[Kyle Stanley]

What exactly does the PR involve? Is it a relatively simple compatibility
patch or something that adds significant amounts of platform-specific code?
The former can be reviewed (and potentially merged) by any core dev
knowledgeable in the areas being changed, but the latter requires long-term
support for the platform.

In order to provide support for a new platform that we don't presently have
support for in CPython, it requires the following:

1) An existing core developer willing to provide long-term support for the
platform. Alternatively, someone else can be granted commit privileges, for
the purpose of maintaining that platform and fixing any issues that come up
in that CI that are specific to it. However, this is done on a case-by-case
basis, as it still requires a decent amount of trust with that individual.

2) A stable buildbot provided, to test against new changes made to CPython.

The purpose of the above is to ensure that any substantial
platform-specific code added to CPython receives proper care; this is
opposed to a large addition of new code for a platform that has nobody to
take care of it when issues come up. Especially as a group of mostly
volunteers (and those paid by their employers to contribute), the latter
wouldn't be sustainable. I personally think it would be fantastic to have
official support for real-time OSs in CPython, but in this ecosystem, it
requires being able to place trust in someone that's willing and adequately
proficient in the related areas to provide long-term support for them.

For more details, see PEP 11:
https://www.python.org/dev/peps/pep-0011/#supporting-platforms.


On Wed, Aug 5, 2020 at 7:44 AM Xin, Peixing 
wrote:

> Hi, Python developers:
>
>
>
> Anyone interested in PR review and merging for VxWorks RTOS? We can give a
> proper pay for that. Contact me if you have interest.
>
>
>
> Thanks,
>
> Peixing
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HATTYX3BWEIPVVUL7FMSNSCFYOBPS6OD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZT2RA7KT7AFMAXTKDQGQOQ6TCUXWE5QT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [OT] I'm unsubscribing from this tire fire (formerly known as python-dev)

[Kyle Stanley]

>
> I'm using my mailer's "ignore thread" feature and counting on the fact
> that the flamers will eventually exhaust themselves (most already have).
>

Yep, not all threads are going to be equally worthwhile for everyone to
read. If a thread is going nowhere productive, the best course of action is
oftentimes to ignore new messages.

Also, on most high traffic public MLs, I think it's all but necessary to
maintain a "kill file" to filter messages from authors that consistently
don't make positive contributions to the discussions. For anyone feeling
burned out that still wants to be involved in the technical discussions, I
would highly recommend considering a similar approach. There are sometimes
cases where an individual may not be necessarily violating the CoC (meaning
there's not much that can be done from a moderation PoV) while still being
generally negative and/or not worthwhile to read messages from on a regular
basis.

On Mon, Jul 6, 2020 at 6:28 AM Ivan Pozdeev via Python-Dev <
python-dev@python.org> wrote:

> I'm using my mailer's "ignore thread" feature and counting on the fact
> that the flamers will eventually exhaust themselves (most already have).
>
> On 06.07.2020 8:41, Christian Heimes wrote:
> > Y'all,
> >
> > trigger warning: strong opinion
> >
> > The Urban Dictionary defines the term "tire fire":
> >
> >A horrifying mess, either literally or figuratively
> >foul-smelling, that seems to last forever.
> >
> > The term describes my current view of python-dev perfectly. It has
> > always been a problematic and mentally draining place for, sometimes
> > even toxic. But the recent PEP-8 discussion trumps every past incident
> > (reference to US politics intended).
> >
> >
> > To every person still replying on the PEP-8 thread:
> >
> >You are making us sick and should be ashamed of yourself!
> >
> > And I don't mean 'sick' in the figurative sense. You are literally
> > hurting people who are spending their free and personal time to develop
> > open source software for you. I know of at least three cases among
> > Python core developers with symptoms like sleep disorder, tremor,
> > anxiety, and panic attacks. One core dev wrote publicly that they were
> > forced to take psychotropic medicine to counter a panic attack after
> > they have read just a few messages.
> >
> >
> > At one point I have even considered to retire from Python core
> > development completely. I'm profoundly disgusted and appalled by the
> > racist attitudes and self-importance of some people as well as an
> > unrelated incident on BPO last week. The two reasons I'm not leaving are
> >   several core developers that I'm happy to call friends and Python
> > communities beyond predominantly male and Western participants on the
> > PEP-8 thread. Communities like PyLadies, PyCon Africa, PyLATAM, and
> > PyCon APAC make me proud and happy to be a member of the Python
> > community. I have met fantastic people at Python and OSS events in the
> > Caribbean, India, and East Europe. I don't want to abandon people I
> > cherish and grew fond of.
> >
> >
> > At least one other core developer has abandoned python-dev last week.
> > Others have stopped participating and posting on python-dev years ago. I
> > will follow their example now.
> >
> > Goodbye
> > Christian
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LR3RWME7NYAVAWGD2ZD5NPZAGL7VVI7K/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> > --
> > Regards,
> > Ivan
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EE2JEIB2CPVLEQR4XTNGFDZWIZJ5LB2L/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HHGATEW2BIJ5AKL6RUVEVWEHL2DWI2VS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Recent PEP-8 change

[Kyle Stanley]

> Basically, it feels like we were lied to.  And if that wasn't bad enough,
to see that Guido accepted that vitriolic commit message and merged it in
... it makes me embarrassed to be a Python supporter.

Only Guido could attest to this, but as someone who spoke in support of the
change, I personally missed the commit message until attention was drawn to
it in the python-ideas thread. When reviewing PRs, I'll admit that I don't
pay a whole lot of attention to the individual commit messages
(particularly extended descriptions); most of my attention is on the actual
changes made. So, perhaps he did the same?

Either way, I don't think I would go as far as to say that it embarrasses
me as a Python contributor. That being said, it did very much feel like it
went in a completely different direction than the PR description, and I'm
uncomfortable with it as well for that reason. So, my vote would be to
amend the commit message based on the description of the PR and proposal
made in python-ideas.

On Tue, Jun 30, 2020 at 8:53 AM Ethan Furman  wrote:

> On 06/30/2020 05:03 AM, Łukasz Langa wrote:
> >
> >> On 30 Jun 2020, at 12:44, Ethan Furman  et...@stoneleaf.us>> wrote:
> >>
> >> Of course I don't know if Keara or Guido knew any of this, but it
> certainly feels to me that the commit message is ostracizing an entire
> family line because they had the misfortune to have the wrong last name.
> In fact, it seems like Strunk & White is making changes to be inclusive in
> its advice -- exactly what I would have thought we wanted on our side ("our
> side" being the diverse and welcoming side).
> >
> > In any case, saying that Keara and Guido mistook the family name of one
> of the authors for skin color feels derogatory.
>
> My apologies, that was not my intent.  As I said, I never knew what it was
> until today (er, yesterday now).
>
> > The commit message clearly is controversial but when you say the change
> itself was unnecessary, consider that English is now a language
> predominantly used outside of USA and Great Britain. Relaxing the
> recommendation to use S & L Standard English in the CPython codebase isn't
> problematic in this sense. That recommendation was largely ignored anyway,
> as core developer voices in the other threads already admitted. So, chaos
> won't ensue. We still want to maintain consistency, as PEP 8 recommends. I
> don't think you have to worry now about seeing organization and
> organisation in the same docstring.
>
> Well, that wouldn't bother me -- as often as not I use non-US-English
> spellings; I just appreciate if it's a correct spelling /somewhere/.
>
> >> That's what it felt like:  betrayal.
> >
> > This entire section of your message is confusing to me. Mind explaining?
> How does a commit message equate stabbing somebody who helped you? What is
> being betrayed in this commit?
>
> The original request for the change had absolutely no hint that the
> current text was racist in any way; then we find out that, apparently,
> we've been harboring white supremacist ideals by prescribing when to use
> apostrophes and commas?  That commit message (not the commit itself) took
> what should have been a simple change and turned into a platform for
> political grandstanding of the worst kind:
>
> - False, as far as I can tell (until given confirming examples from the
> S text)
> - Only colored people are mentioned (and other /native English speakers/)
> - Zero mention of non-native English speakers
>
> Basically, it feels like we were lied to.  And if that wasn't bad enough,
> to see that Guido accepted that vitriolic commit message and merged it in
> ... it makes me embarrassed to be a Python supporter.
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/T55AGV7XPQ7YHU2VFIBHAWUZHEMZ43R3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DOM5V5QSKFJSD3FKVXIF4C4X3AFE6QNR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[Kyle Stanley]

I tend to keep out of these types of discussions because they have a
tendency to be rather polarizing, and when introduced in an unrelated
environment (such as python-ideas or python-dev), tend to do nothing other
than set people against each other. But, after the above message, I feel
obligated to respond. Also, keep in mind that I'm stating this as someone
who fully agreed with the change made (as mentioned earlier in the thread).

Thomas Wouters wrote:
> I'm not sure who 'our' is in this sentence, but I'm certainly not glad
Python ever took any of your patches

Regardless of thoughts on the above matter, this is absolutely uncalled for
Thomas, especially coming from someone who has served on the PSF board and
was elected for the upcoming term. As someone who voted for you in the
recent election thinking you would represent the core development team
well, the above statement does make me question that to some degree. I hope
this situation is an outlier, and not indicative of how problems will be
addressed in the future.

I for one greatly appreciate the contributions by Giampaolo. Our CI would
be nowhere near where it is today without his help, not to mention the
countless refleaks, bug fixes, and expertise provided on a vast number of
issues. I could go on, but quite frankly it's not especially relevant,
because I'd say the same for any other member of the core team or active
contributor.

I really can't imagine in any way how this statement isn't a clear
violation of the CoC (https://www.python.org/psf/conduct/).

> *Acknowledging time and effort*. We're respectful of the volunteer
efforts that permeate the Python community. We're thoughtful when
addressing the efforts of others, keeping in mind that often times the
labor was completed simply for the good of the community.

The statement completely disregards the time and effort put into
Giampaolo's contributions to Python.

> Showing empathy towards other community members. We're attentive in our
communications, whether in person or online, and *we're tactful when
approaching differing views*.

> *Being respectful.* We're respectful of others, their positions, their
skills, their commitments, and their efforts.

The statement is clearly not tactful or respectful of his differing
viewpoint.

> I'm not sure how much more clear python-dev and the PSF could have been
that this is true. Your complaints of "racism against whites" here haven't
gone unheard, unfortunately. They reflect very badly on you. They are
incredibly harmful to many python-dev members and the Python community as a
whole, and they reflect very badly on all of us.

I can see how the "racism against whites" argument could come across as
being non-inclusive towards others, particularly because these arguments
are often put forth by white supremacists for ill-intended purposes.
However, telling someone who has devoted countless hours towards
contributing to Python that "I'm certainly not glad Python ever took any of
your patches" is pretty much the exact opposite of a tactful way to
approach this issue. Even assuming that Giampolo's views are in violation
of the CoC (which I'm not saying they are or aren't), the way to handle it
is most certainly not also by violating the CoC against him.

If those views reflect badly on us, I think it reflects even worse on us
that we have to resort to telling people that their contributions are not
valued because they have a different point of view, even if that view is
offensive to others. Are we not capable of having this discussion in a
civil manner? I realize that this is a topic that is dear to many people
(including myself, as someone who is politically progressive), but that
does not make this sort of exchange acceptable.



On Tue, Jun 30, 2020 at 5:59 AM Thomas Wouters  wrote:

>
>
> On Tue, Jun 30, 2020 at 4:28 AM Giampaolo Rodola' 
> wrote:
>
>> This is not about the commit message. It’s way more than that. It's been
>> going on non-stop and got increasingly worse since at least the preparation
>> of the Python elections ~2 years ago. It is not normal what is going on
>> here. People are scared. And it is pretty much guaranteed that this is not
>> gonna be the last occurrence of it. On the horizon we have other
>> language-related controversies like "whitelist" / "blacklist", renaming
>> "master" to "main" in GIT, and who knows what else (maybe "whitespace"? or
>> @property?).
>>
>
> I don't have words for the irony of complaining about changing words while
> objecting to the wording in a commit message. Especially considering the
> commit message isn't nearly as visible as the places that people have
> actually been fixing things like master/slave.
>
>
>> And every time that's gonna happen the motivation is gonna be about white
>> supremacy/privilege/guilt etc. Because it's always about that, and we'll be
>> having this discussion once again. On one hand Python gladly takes our
>> patches and everything is smooth,
>>
>
> I'm not sure who 

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Kyle Stanley]

Thank you very much to Brandt, Tobias, Ivan, Guido, and Talin for the
extensive work on this PEP. The attention to detail and effort that
went into establishing the real-world usefulness of this feature (such
as the many excellent examples and code analysis) helps a lot to
justify the complexity of the proposed pattern matching feature(s).
The concern about the added complexity to the language is certainly
reasonable (particularly in the sphere of researchers and other
non-developers), but I think this is a case where practicality will
ultimately win.

Overall, I am +1 for this PEP. That being said, I would like to state
a few opinions (with #2 being my strongest one):

1) I was initially in agreement about the usage "else:" instead of
"match _:", but upon further consideration, I don't think "else:"
holds up very well in the context of pattern matching. Although it
could be confusing at a first glance (admittedly, it threw me off at
first), an underscore makes far more sense as a wildcard match;
especially considering the existing precedent.

2) Regarding the constant value pattern semantics, I'm okay with the
usage of the "." in general, but I completely agree with several
others that it's rather difficult to read when there's a leading
period with a single word, e.g. ".CONSTANT". To some degree, this
could probably be less problematic with some reasonably good syntax
highlighting to draw attention to the leading period.

However, I don't think it should be at all necessary for people to
rely on syntax highlighting to be able to clearly see something that's
part of a core Python language feature. It seems especially
detrimental for those with visual impairment. As someone with
relatively poor eye-sight who typically has to blow up the font size
for my code to be readable (and often finds syntax highlighting to be
distracting), I'm not really looking forward to squinting for missed
leading periods when it was intended to refer to a constant reference.
Even if it's a relatively uncommon case, with a core feature, it's
bound to happen enough to cause some headaches.

>From the "Rejected Ideas" section:

> Introduce a special symbol, for example $ or ^ to indicate that a given name 
> is a constant to be matched against, not to be assigned to:
>
> FOO = 1
> value = 0
>
> match value:
>case $FOO:  # This would not be matched
>...
>case BAR:  # This would be matched
>...
>
> The problem with this approach is that introducing a new syntax for such 
> narrow use-case is probably an overkill.

I can certainly understand that it seems overkill to have a separate
symbol for this, but in my biased opinion, I think it's worth a
stronger consideration from the perspective of those with some degree
of visual impairment. I don't have a strong opinion about the specific
larger symbol that should be used instead, but either of "$" or "^"
would be perfectly fine by me. I'd be on-board with anything that
doesn't have a strong existing purpose in the language.

3) Regarding the 6 before vs. after examples provided by Guido, I have
some thoughts on the last one:

> Original:
>
> def flatten(self) -> Rhs:
> # If it's a single parenthesized group, flatten it.
> rhs = self.rhs
> if (
> not self.is_loop()
> and len(rhs.alts) == 1
> and len(rhs.alts[0].items) == 1
> and isinstance(rhs.alts[0].items[0].item, Group)
> ):
> rhs = rhs.alts[0].items[0].item.rhs
> return rhs
>
> Converted (note that I had to name the classes Alt and NamedItem, which are 
> anonymous in the original):
>
> def flatten(self) -> Rhs:
> # If it's a single parenthesized group, flatten it.
> rhs = self.rhs
> if not self.is_loop():
> match rhs.alts:
> case [Alt(items=[NamedItem(item=Group(rhs=r))])]:
> rhs = r
> return rhs

I think part of it is just that I tend to find anything that has 4+
layers deep of nested parentheses and/or brackets to be a bit
difficult to mentally parse, but my reaction to seeing something like
"case [Alt(items=[NamedItem(item=Group(rhs=r))])]:" in the wild
without anything to compare it to would probably be o_0. I definitely
find the 4-part conditional in the "Original" version to be a lot
easier to quickly understand, even if it's a bit redundant and
requires some guard checks. So IMHO, that specific example isn't
particularly convincing.

That being said, I found the other 5 examples to be very easy to
understand, with the second one being the one to really win me over.
The proposed class matching is a drastic improvement over a massive
wall of "if/elif isinstance(...):" conditionals, and I really like the
way it lines up visually with the constants. Also, in time, I could
very well change my mind about the last example after getting more
used to the proposed syntax.



On Tue, Jun 23, 2020 at 12:04 PM Guido van Rossum  wrote:
>
> I'm happy to present a new PEP for the python-dev community to 

[Python-Dev] Re: When can we remove wchar_t* cache from string?

[Kyle Stanley]

> I'm sorry, I was wrong.  Py_DEPRECATED(3.3) is commented out for some
APIs.
So Python 3.8 doesn't show warning for them.

Ah, no problem. Thanks for checking up on that.

> I still think 2 years are enough to removal.

Hmm, okay. At the least though, it does mean we have to be a bit more
vigilant in ensuring that everyone has had a chance to migrate from those
APIs, and delaying the removal if not.


On Sun, Jun 14, 2020 at 9:34 PM Inada Naoki  wrote:

> On Sat, Jun 13, 2020 at 8:20 PM Inada Naoki 
> wrote:
> >
> > 2020年6月13日(土) 20:12 Kyle Stanley :
> >>
> >> > Additionally, raise DeprecationWarning runtime when these APIs are
> used.
> >>
> >> So, just to clarify, current usage of these 7 unicode APIs does not
> emit any warnings and would only start doing so in 3.10?
> >
> > They have been deprecated in C already.  Compiler emits warning.
> >
> > This additional proposal is adding runtime warning before removal.
> >
>
> I'm sorry, I was wrong.  Py_DEPRECATED(3.3) is commented out for some APIs.
> So Python 3.8 doesn't show warning for them.
> I want to uncomment them in Python 3.9.
> https://github.com/python/cpython/pull/20878
>
> As far as I grepped, most of PyPI packages use deprecated APIs because
> Cython generates it.
> Updating Cython will fix them.
> Some of them are straightforward and I have created an issue or sent
> pull request already.
>
> A few projects, pyScss and Genshi are not straightforward.  But it is
> not too hard and I will help them.
>
> I still think 2 years are enough to removal.
>
> Regards,
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IKT3F2Q2CWFCRYWBKQYAL2DOZJFKQK2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When can we remove wchar_t* cache from string?

[Kyle Stanley]

> They have been deprecated in C already.  Compiler emits warning.

> This additional proposal is adding runtime warning before removal.

Oh, sorry. I misunderstood the previous statement then. In that case, I
think scheduling the removal for 3.11 is perfectly reasonable if the
compiler warnings have already been in place. +1.


On Sat, Jun 13, 2020 at 7:20 AM Inada Naoki  wrote:

>
>
> 2020年6月13日(土) 20:12 Kyle Stanley :
>
>> > Additionally, raise DeprecationWarning runtime when these APIs are used.
>>
>> So, just to clarify, current usage of these 7 unicode APIs does not emit
>> any warnings and would only start doing so in 3.10?
>>
>
> They have been deprecated in C already.  Compiler emits warning.
>
> This additional proposal is adding runtime warning before removal.
>
>
> In this case, it might be okay to remove in 3.11 since they've been
>> deprecated for an exceptionally long period and appear to have a clear
>> transition path. But, 3.12 would be safer for removal, and I don't think it
>> would be much of an additional burden on our end to keep them around for
>> one extra version.
>>
>
> I am trying to find and remove use of theses APIs in PyPI packages.
> We will postpone the removal if the migration is slow.
> But let's set the goal to 3.11 for now.
>
>>
>>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LMGTX4MWJ3BDV3VGAGQT3VAWBCVI35RD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: When can we remove wchar_t* cache from string?

[Kyle Stanley]

> Additionally, raise DeprecationWarning runtime when these APIs are used.

So, just to clarify, current usage of these 7 unicode APIs does not emit
any warnings and would only start doing so in 3.10? If so, I think we may
want to consider giving users until 3.12 until they're removed. Especially
with the shortened yearly release cadence, a single version between
deprecation warning and removal feels a bit short. Even if they've already
been announced as deprecated in whatsnew and the documentation since 3.3,
it's very possible for it to have been missed.

In this case, it might be okay to remove in 3.11 since they've been
deprecated for an exceptionally long period and appear to have a clear
transition path. But, 3.12 would be safer for removal, and I don't think it
would be much of an additional burden on our end to keep them around for
one extra version.

Another option might be to proceed with the 3.11 removal, and simply delay
it to 3.12 if we receive significant complaints of breakage in 3.11 to give
users some extra time to address it. As far as I'm aware, this isn't
typically done, but I think it would be more than reasonable in this
scenario (assuming the deprecation warnings are just being introduced in
3.10).

On Sat, Jun 13, 2020 at 6:46 AM Inada Naoki  wrote:

> On Fri, Jun 12, 2020 at 5:32 PM Inada Naoki 
> wrote:
> >
> >
> > My proposal is, schedule the removal on Python 3.11.  But we will
> postpone
> > the removal if we can not remove its usage until it.
> >
>
> Additionally, raise DeprecationWarning runtime when these APIs are used.
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3L456JBACI2UBGKAXYNVBOG7RVEREXPC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DLFT2BVN4BYV2I4KRZW6B2ZSS7ZVLVG5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [python-committers] Please welcome our next Release Manager, Pablo!

[Kyle Stanley]

> In light of the release of Python 3.9b1, let’s take a moment to celebrate
all the great work that our Python 3.8 and 3.9 release manager Łukasz has
done.

Thank you so much to Łukasz for a fantastic 3.8 release, and for the smooth
transition into 3.9 beta. :-)

> Please join me in welcoming Pablo in his new role!

Congrats, Pablo! With all of your incredible work on CPython's continuous
integration, I think you'll be a great release manager!

On Tue, May 19, 2020 at 6:54 PM Barry Warsaw  wrote:

> In light of the release of Python 3.9b1, let’s take a moment to celebrate
> all the great work that our Python 3.8 and 3.9 release manager Łukasz has
> done.  The role of Python Release Manager is hugely important to each
> successful release, and it can be a lot of work, often unseen and thankless
> to shepherd a new Python version through its first alpha release to its
> last security release.  With all of your immeasurable help, the Release
> Manager ensures solid, feature-full releases that the entire Python
> community eagerly awaits.
>
> Łukasz carries on the fine tradition of all of our past release managers,
> and now that his second release has entered beta phase, I’m very happy to
> announce our next Release Manager, for Python 3.10 and 3.11: Pablo Galindo
> Salgado!
>
> Since becoming a core developer in 2018, Pablo has contributed
> significantly to Python.  With the change to an annual release cycle (PEP
> 602, authored by Łukasz), the time commitment for release managers has been
> reduced as well, and we will continue to look for ways to make the
> selection process for release managers more transparent and accessible.  I
> know that in addition to admirably managing the releases for 3.10 and 3.11,
> Pablo will also help to continually improve the process of selecting and
> serving as release manager.
>
> Please join me in welcoming Pablo in his new role!
>
> Cheers,
> -Barry
>
> ___
> python-committers mailing list -- python-committ...@python.org
> To unsubscribe send an email to python-committers-le...@python.org
> https://mail.python.org/mailman3/lists/python-committers.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-committ...@python.org/message/44TLJO5YX6XYM4ICWSHMBMCKPBBQQP5S/
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J3CGQNJ2UH3RPFJ2IEVGERTIRAMGXOLI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecate os.removedirs() and os.renames()

[Kyle Stanley]

Serhiy Storchaka wrote:
> I propose to deprecate these functions and remove them in future Python
versions.

+1, assuming the deprecation lasts for at least two versions and the
available alternatives are explicitly mentioned in the What's New
entry (for both the version they're initially deprecated in and the
one they become removed in). Although, I suspect that the deprecation
may need to be longer than two versions from possible breakage in
older libraries or legacy code. They don't seem at all common compared
to shutil.move() and shutil.rmtree(), but I do vaguely recall seeing
some usage of os.renames() and os.removedirs() in third party code.

On Thu, May 7, 2020 at 4:06 PM Serhiy Storchaka  wrote:
>
> It seems to me that os.removedirs() and os.renames() was added just for
> symmetry with os.makedirs(). All three functions have similar structure
> and was added in the same commit. Seems they were initially code
> examples of using some os.path and os functions.
>
> Unlike to quite popular os.makedirs(), os.removedirs() and os.renames()
> are not used in the stdlib and rarely used in third party code.
> os.removedirs() is considered as an opposite to os.makedirs(), and
> os.renames() is a combination of os.makedirs(), os.rename() and
> os.removedirs(). The problems with them are:
>
> 1. They do not remove directory if any file or other subdirectory is
> left. They just stop removing and return success. ZTo the user it looks
> like they do not work as expected, but he need to test the existence of
> directory explicitly to check this.
>
> 2. They can remove more than expected. If the parent directory was empty
> before calling os.makedirs(), the following os.removedirs() will remove
> not just the newly created directories, but the parent directory, and
> its parent if it contained a single directory, and so on.
>
> os.removedirs() is not an opposite to os.makedirs(). It can remove less
> or more, and you have no control on how much it will remove. It is
> better to use shutil.rmtree().
>
> os.renames() correspondingly can be replaced by os.rename() or
> shutil.move(), with possible addition of os.makedirs() and
> shutil.rmtree() if needed.
>
> I propose to deprecate these functions and remove them in future Python
> versions.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/MWUFGKT43L3KJXN33DLTYN6OLDB6GP45/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KDW3D7HAV7EEMSA2DJNR7NQRPG6TEJTB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Comments on PEP 554 (Multiple Interpreters in the Stdlib)

[Kyle Stanley]

Mark Shannon wrote:
> If `run()` can raise
> an exception, why not let it return values?

If there's not an implementation detail that makes this impractical,
I'd like to give my +1 on the `Interpreter.run()` method returning
values. From a usability perspective, it seems incredibly convenient
to have the ability to call a function in a subinterpreter, and then
directly get the return value instead of having to send the result
through a channel (for more simple use cases).

Also, not that the API for subinterpreters needs to be at all similar
to asyncio, but it would be consistent with `asyncio.run()` with
regards to being able to return values. Although one could certainly
argue that `asyncio.run()` and `Interpreter.run()` will have
significantly different use cases; with `asyncio.run()` being intended
as a primary entry point for a program, and `Interpreter.run()` being
used to execute arbitrary code in a single interpreter.

On Tue, Apr 21, 2020 at 12:45 PM Mark Shannon  wrote:
>
> Hi,
>
> I'm generally in favour of PEP 554, but I don't think it is ready to be
> accepted in its current form.
>
> My main objection is that without per-subinterpeter GILs (SILs?) PEP 554
> provides no value over threading or multi-processing.
> Multi-processing provides true parallelism and threads provide shared
> memory concurrency.
>
> If per-subinterpeter GILs are possible then, and only then,
> sub-interpreters will provide true parallelism and (limited) shared
> memory concurrency.
>
> The problem is that we don't know whether we can implement
> per-subinterpeter GILs without too large a negative performance impact.
> I think we can, but we can't say so for certain.
>
> So, IMO, we should not accept PEP 554 until we know for sure that
> per-subinterpeter GILs can be implemented efficiently.
>
>
>
> Detailed critique
> -
>
> I don't see how `list_all()` can be both safe and accurate. The Java
> equivalent makes no guarantees of accuracy.
> Attempting to lock the list is likely to lead to deadlock and not
> locking it will lead to races; potentially dangerous ones.
> I think it would be best to drop this.
>
> `list_all_channels()`. See `list_all()` above.
>
> `.destroy()` is either misleading or unsafe.
> What does this do?
>
>  >>> is.destroy()
>  >>> is.run()
>
> If `run()` raises an exception then the interpreter must exist. Rename
> to `close()` perhaps?
>
> `Channel.interpreters` see `list_all()` and `list_all_channels()` above.
>
> How does `is_shareable()` work? Are you proposing some mechanism to
> transfer an object from one sub-interpreter to another? How would that
> work? If objects are not shared but serialized, why not use marshal or
> pickle instead of inventing a third serialization protocol?
>
> It would be clearer if channels only dealt with simple, contiguous
> binary data. As it stands the PEP doesn't state what form the received
> object will take.
> Once channels supporting the transfer of bytes objects work, then it is
> simple to pass more complex objects using pickle or marshal.
>
> Channels need a more detailed description of their lifespan. Ideally a
> state machine.
> For example:
> How does an interpreter detach from the receiving end of a channel that
> is never empty?
> What happens if an interpreter deletes the last reference to a non-empty
> channel? On the receiving end, or on the sending end?
>
> Isn't the lack of buffering in channels a recipe for deadlocks?
>
> What is the mechanism for reliably copying exceptions from one
> sub-interpreter to another in the `run()` method? If `run()` can raise
> an exception, why not let it return values?
>
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/ZSE2G37E24YYLNMQKOQSBM46F7KLAOZF/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YSZBQEES7LCBANVIRIUXSKDHZGL3Q2F6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 554 comments

[Kyle Stanley]

Eric Snow wrote:
> We will mark it "provisional" in the docs, which I expect will include
> info on what that means and why it is provisional.

If you'd like an example format for marking a section of the docs as
provisional w/ reST, something like this at the top should suffice
(with perhaps something more specific to the subinterpreters module):


.. note::
This section of the documentation and all of its members have been
added *provisionally*. For more details, see :term:`provisional api`.


:term:`provisional api` generates a link to
https://docs.python.org/3/glossary.html#term-provisional-api.

On Tue, Apr 21, 2020 at 12:09 PM Eric Snow  wrote:
>
> On Tue, Apr 21, 2020 at 7:25 AM Victor Stinner  wrote:
> > Would it make sense to start by adding the module as a private
> > "_subinterpreters" module but document it? The "_" prefix would be a
> > reminder that "hey! it's experimental, there is a no backward
> > compatibility warranty there".
>
> I would expect a leading underscore to be confusing (as well as
> conflicting with the name of the low-level module).  if we did
> anything then it would probably make more sense to name the module
> something like "interpreters_experimental".  However, I'm not sure
> that offers that much benefit.
>
> > We can also add a big warning in the documentation.
>
> We will mark it "provisional" in the docs, which I expect will include
> info on what that means and why it is provisional.
>
> -eric
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/LB7ACPFAT3QU5YUPIFWH4NHB7YYVJXQG/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TSIV524HRFGTKBPPV2IMO5J2U4QGDDKQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Kyle Stanley]

Raymond Hettinger wrote:
>  Yes, I get that.  Just want to point-out that working with heavily
nested dictionaries (typical for JSON) is no fun with square brackets and
quotation marks.

I can certainly agree with that sentiment, especially when working with
something like GraphQL that tends to return deeply nested JSON objects.
Repeatedly using [''] can get quite tiresome (and not look particularly
great) with something like this:

```
for pr_edge in pr_json['data']['user']['pullRequests']['edges']:
 for comment_edge in pr_edge['node']['comments']['edges']:
  commenter = comment_edge['node']['author']['login']
  ...
```

(Extracted from a personal side-project I worked on last year)

On Wed, Apr 15, 2020 at 12:10 AM Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:

> [GvR]
> > We should not try to import JavaScript's object model into Python.
>
> Yes, I get that.  Just want to point-out that working with heavily nested
> dictionaries (typical for JSON) is no fun with square brackets and
> quotation marks.
>
>
> Raymond
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/G5SJKRQ7S5VY3JKLAVOTCCA7RSDUNWXS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DUL6QQ4VPGUSDKF4XBUTRTKJNRSOGY62/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Kyle Stanley]

Guido van Rossum wrote:
> I've seen this pattern a lot at a past employer, and despite the obvious
convenience I've come to see it as an anti-pattern: for people expecting
Python semantics it's quite surprising to read code that writes foo.bar and
then reads back foo['bar'].

Would it be significantly less surprising if it were to be exclusively
allowed for the JSON module rather than anything that accepts custom dicts?
To me, that seems like the most useful and intuitive location for the dot
notation, and it would keep the Python semantics intact everywhere else. I
think it's fairly unlikely that most users of JSON would be surprised by
it, but I can see that it could be surprising elsewhere.

On Tue, Apr 14, 2020 at 11:33 PM Guido van Rossum  wrote:

> I've seen this pattern a lot at a past employer, and despite the obvious
> convenience I've come to see it as an anti-pattern: for people expecting
> Python semantics it's quite surprising to read code that writes foo.bar and
> then reads back foo['bar']. We should not try to import JavaScript's object
> model into Python.
>
> On Tue, Apr 14, 2020 at 8:06 PM Raymond Hettinger <
> raymond.hettin...@gmail.com> wrote:
>
>> SimpleNamespace() is really good at giving attribute style-access. I
>> would like to make that functionality available to the JSON module (or just
>> about anything else that accepts a custom dict) by adding the magic methods
>> for mappings so that this works:
>>
>>  catalog = json.load(f, object_hook=SimpleNamespace)
>>
>>  print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity'])
>> # currently possible with dict()
>>  print(catalog.clothing.mens.shoes.extra_wide.quantity])
>># proposed with SimpleNamespace()
>>  print(catalog.clothing.boys['3t'].tops.quantity
>>  # would also be supported
>>
>> I've already seen something like this in production; however, people are
>> having to write custom subclasses to do it.  This is kind of bummer because
>> the custom subclasses are a pain to write, are non-standard, and are
>> generally somewhat slow.  I would like to see a high-quality version this
>> made more broadly available.
>>
>> The core idea is keep the simple attribute access but make it easier to
>> load data programmatically:
>>
>> >>> ns = SimpleNamespace(roses='red', violets='blue')
>> >>> thing = input()
>> sugar
>> >>> quality = input()
>> sweet
>> >>> setattr(ns, thing, quality)# current
>> >>> ns['sugar'] = 'sweet'   # proposed
>>
>> If the PEP 584 __ior__ method were supported, updating a SimpleNamespace
>> would be much cleaner:
>>
>>   ns |= some_dict
>>
>> I posted an issue on the tracker: https://bugs.python.org/issue40284 .
>> There was a suggestion to create a different type for this, but I don't see
>> the point in substantially duplicating everything SimpleNamespace already
>> does just so we can add some supporting dunder methods.   Please add more
>> commentary so we can figure-out the best way to offer this powerful
>> functionality.
>>
>>
>> Raymond
>>
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/JOMND56PJGRN7FQQLLCWONE5Z7R2EKXW/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EJ6XJW6SZ23V2RJTSPPYT4Z3FB7BMQGO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5B6ACDDG5HR2NY32SZY2GPFLNKOVBMDJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Kyle Stanley]

Looking over the commit history for the PR (
https://github.com/python/cpython/pull/18239/commits), it looks like that
specific Azure Pipelines failure did not start occurring until
upstream/master was merged into the PR branch (
https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea77060547de6d8445877e820e).
Therefore, I suspect that the failure is very likely unrelated to the PR;
instead either an intermittent failure that was merged into master recently
or a possible issue on Azure's end. For now, I'd suggest closing and
re-opening the PR again tomorrow to see if the failure still occurs.

Note: I'm also seeing the same exact failure occur in the following
separate CPython PRs that were opened recently:

https://github.com/python/cpython/pull/19403
https://github.com/python/cpython/pull/19402
https://github.com/python/cpython/pull/19399

Seeing as it was also occurring in entirely unrelated PRs, it seems to be
unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure,
so I'll open a new one for it.

On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum  wrote:

> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
> trusty assistant who has a Windows laptop couldn't reproduce the failure.
> Can I buy a hint from someone? Steve?
>
> The relevant failure output is:
>
> ==
> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
> [WinDLL('_sqlite3.dll', winmode=0)]
> --
> Traceback (most recent call last):
>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
> subprocess.check_output(
>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command
> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
> status 1.
>
> --
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HDLEJCO34DGCHADVEJD3ZWUH4LYJVHES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enum._convert should change __repr__ and/or __str__ to use module name instead of class name

[Kyle Stanley]

Ivan Pozdeev wrote:
> More information is not better if that information is harmful rather than
helpful.

While that argument does apply in some cases, I'd have to very much
disagree that "" is harmful in comparison to just "1";
it clearly shows the value on the right side of the colon. As for the
source location for "socket.AF_UNIX" not being clear, that may be the case
the first time you encounter it or haven't seen it in a while, but it's
clear from a quick search of the help page.

IMO, the *full* location shouldn't have to appear every time you want to
see the repr; it would add unneeded noise. If you see the repr and it's not
clear, consult the help page and/or docs for more information. The repr
should be succinct.

> And help() doesn't even show the value!

Sorry, but this is simply untrue. If you scroll down on the help page, or
search via "/" for "AF_UNIX", you'll encounter the following:

|  AF_UNIX = 

(which is likely what Ethan was referring to)

If it wasn't clear, when you use "help(socket.AF_UNIX)", it brings up the
AddressFamily help page (as indicated in the title), not a dedicated page
for "socket.AF_UNIX". You can also find the information present in the
"help(socket)" page if you search for "AF_UNIX".

> Pardon me if I misunderstood you, but I also find the offhand remarks:
"adding this and that to your debugging list would also be, um,
> helpful" -- highly condescending and insulting and showing that you
probably don't do much problem diagnostic and don't care at all about
> anyone who does -- i.e. about software developers and maintainers, your
core audience.

>From my perspective, Ethan pointed out "help()" because it's very
frequently neglected, even by experienced devs; not at all to be
condescending towards you or insulting of your experience. However,
explicitly claiming that "you probably don't do much problem diagnostic and
don't care at all about anyone who does" is *directly* insulting and
doesn't add anything constructive to the discussion. Insulting the other
party, especially when they're responding to you voluntarily and not paid
to do so, makes them far less likely to consider your side or even respond
at all.

Rather than the extreme of "either you agree with me or don't have
experience with this", is it not possible that you simply disagree and have
different preferences in what makes more for more optimal debugging
information?

> All the while yourself being a Python core developer (if your
StackOverflow profile is to be believed) -- a spokesperson for the dev team
> and a shining example of what every Python developer should strive to be.

I'm not sure what the purpose of the "if your StackOverflow profile is to
be believed" part is, but the list of Python core developers can be found
here: https://devguide.python.org/developers/ (where "Ethan Furman" is
clearly listed).

> If this is any representation of the prevalent attitude in the core team
now, I'm probably wasting my time arguing here as well as
> supporting Python in general since our values clearly don't match anymore.

I'm sorry that you feel that way. But if you considered the discussion to
be a waste of time, then I don't think python-dev (or similar OSS dev
communities) will be a very productive experience for you. Also, just to be
clear: the opinion of one or a few core devs does not represent the
opinions or values of the entire core team. The core team is made up of
~100 current individuals with differing opinions, areas of expertise, and
perspectives.

On Fri, Mar 27, 2020 at 8:27 PM Ivan Pozdeev via Python-Dev <
python-dev@python.org> wrote:

> On 26.03.2020 19:24, Ethan Furman wrote:
> > On 03/25/2020 06:53 PM, Ivan Pozdeev via Python-Dev wrote:
> >
> >> A diagnostic is always done by the same algorithm:
> >>
> >> 1) Identify the exact place in code where the problem manifests itself
> >> 2) Examine the state of the program at that moment to find out which if
> the values are wrong
> >> 3) Track the wrong value back to its origin
> >>
> >> Seeing an accurate repr() is required for step 2) -- to immediately
> detect if it's wrong or not.
> >
> > Before Enum, the repr for socket.AF_UNIX was:
> >
> > 1
> >
> > Not very useful for debugging.
>
> On the contrary, it's perfect. I know everything I need: it's an integer
> constant, end of story.
>
> When examining some live object, I would probably see that "1" and won't
> even need to look anything up to know what I'm dealing with.
>
> >
> >> If a repr() points to a reference rather than definition, I don't
> really know anything about the object that I'm looking at: that
> >> reference could as well be wrong, or could have been wrong at any
> moment in the past. I.e. such a repr() has negative informativity.
> >
> > Whether it's
> >
> >   
> >
> > or
> >
> >   
> >
> > it has light-years more information than it used to  (okay, maybe only
> light-minutes  ;) )
> >
> More information is not better if that information is harmful rather than
> helpful.

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

[Kyle Stanley]

> -1 on "cut*" (feels too much like what .partition() does)
> -0 on "trim*" (this is the name used in .NET instead of "strip", so I
> foresee new confusion)
> +1 on "remove*" (because this is exactly what it does)

I'm also most strongly in favor of "remove*" (out of the above options).
I'm opposed to cut*, mainly because it's too ambiguous in comparison to
other options such as "remove*" and "replace*", which would do a much
better job of explaining the operation performed.

Without the .NET conflict, I would normally be +1 on "trim*" as well; with
it in mind though, I'd lower it down to +0. Personally, I don't consider a
conflict in a different ecosystem enough to lower it down to -0, but it
still has some influence on my preference.

So far, the consensus seems to be in favor of "remove*" with several +1s
and no arguments against it (as far as I can tell), whereas the other
options have been rather controversial.

On Tue, Mar 24, 2020 at 3:38 PM Steve Dower  wrote:

> On 24Mar2020 1849, Brett Cannon wrote:
> > -1 on "cut*" because my brain keeps reading it as "cute".
> > +1 on "trim*" as it is clear what's going on and no confusion with
> preexisting methods.
> > +1 on "remove*" for the same reasons as "trim*".
> >
> > And if no consensus is reached in this thread for a name I would assume
> the SC is going to ultimately decide on the name if the PEP is accepted as
> the burden of being known as "the person who chose _those_ method names on
> str" is more than any one person should have bear. ;)
>
> -1 on "cut*" (feels too much like what .partition() does)
> -0 on "trim*" (this is the name used in .NET instead of "strip", so I
> foresee new confusion)
> +1 on "remove*" (because this is exactly what it does)
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KVU75BNXIUBIOYM6ZJSPZSKNRS7Y6CYU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EHK7WWVFUOMSD7NJDLOM7S5JKXK6WE3Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

[Kyle Stanley]

> In this case, being in line with the existing string API method names
take priority over PEP 8, e.g. splitlines, startswith, endswith,
splitlines, etc.

Oops, I just realized that I wrote "splitlines" twice there. I guess that
goes to show how much I use that specific method in comparison to the
others, but the point still stands. Here's a more comprehensive set of
existing string methods to better demonstrate it (Python 3.8.2):

>>> [m for m in dir(str) if not m.startswith('_')]
['capitalize', 'casefold', 'center', 'count', 'encode', 'endswith',
'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum',
'isalpha', 'isascii', 'isdecimal', 'isdigit', 'isidentifier', 'islower',
'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join',
'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind',
'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

On Sun, Mar 22, 2020 at 12:17 AM Kyle Stanley  wrote:

> Ivan Pozdeez wrote:
> > I must note that names conforming to
> https://www.python.org/dev/peps/pep-0008/#function-and-variable-names
> would be "strip_prefix" and "strip_suffix".
>
> In this case, being in line with the existing string API method names take
> priority over PEP 8, e.g. splitlines, startswith, endswith, splitlines,
> etc. Although I agree that an underscore would probably be a bit easier to
> read here, it would be rather confusing to randomly swap between the naming
> convention for the same API. The benefit gained in *slightly *easier
> readability wouldn't make up for the headache IMO.
>
> On Sun, Mar 22, 2020 at 12:13 AM Ivan Pozdeev via Python-Dev <
> python-dev@python.org> wrote:
>
>> On 22.03.2020 6:38, Guido van Rossum wrote:
>>
>> On Sat, Mar 21, 2020 at 6:46 PM Nick Coghlan  wrote:
>>
>>> On Sat., 21 Mar. 2020, 11:19 am Nathaniel Smith,  wrote:
>>>
>>>> On Fri, Mar 20, 2020 at 11:54 AM Dennis Sweeney
>>>>  wrote:
>>>> > This is a proposal to add two new methods, ``cutprefix`` and
>>>> > ``cutsuffix``, to the APIs of Python's various string objects.
>>>>
>>>> The names should use "start" and "end" instead of "prefix" and
>>>> "suffix", to reduce the jargon factor and for consistency with
>>>> startswith/endswith.
>>>>
>>>
>>> This would also be more consistent with startswith() & endswith(). (For
>>> folks querying this: the relevant domain here is "str builtin method
>>> names", and we already use startswith/endswith there, not
>>> hasprefix/hassuffix. The most challenging relevant audience for new str
>>> builtin method *names* is also 10 year olds learning to program in school,
>>> not adults reading the documentation)
>>>
>>
>> To my language sense, hasprefix/hassuffix are horrible compared to
>> startswith/endswith. If you were to talk about this kind of condition using
>> English instead of Python, you wouldn't say "if x has prefix y", you'd say
>> "if x starts with y". (I doubt any programming language uses hasPrefix or
>> has_prefix for this, making it a strawman.)
>>
>> *But*, what would you say if you wanted to express the idea or removing
>> something from the start or end? It's pretty verbose to say "remove y from
>> the end of x", and it's not easy to translate that into a method name.
>> x.removefromend(y)? Blech! And x.removeend(y) has the double 'e', which
>> confuses the reader.
>>
>> The thing is that it's hard to translate "starts" (a verb) into a noun --
>> the "start" of something is its very beginning (i.e., in Python, position
>> zero), while a "prefix" is a noun that specifically describes an initial
>> substring (and I'm glad we don't have to use *that* :-).
>>
>>
>>> I think the concern about stripstart() & stripend() working with
>>> substrings, while strip/lstrip/rstrip work with character sets, is valid,
>>> but I also share the concern about introducing "cut" as yet another verb to
>>> learn in the already wide string API.
>>>
>>
>> It's not great, and I actually think that "stripprefix" and "stripsuffix"
>> are reasonable. (I found that in Go, everything we call "strip" is called
>> "Trim", and there are "TrimPrefix" and "TrimSuffix" functions that
>> correspond to the PEP 616 functions.)
>>
>> I must note that names conformin

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

[Kyle Stanley]

Ivan Pozdeez wrote:
> I must note that names conforming to
https://www.python.org/dev/peps/pep-0008/#function-and-variable-names would
be "strip_prefix" and "strip_suffix".

In this case, being in line with the existing string API method names take
priority over PEP 8, e.g. splitlines, startswith, endswith, splitlines,
etc. Although I agree that an underscore would probably be a bit easier to
read here, it would be rather confusing to randomly swap between the naming
convention for the same API. The benefit gained in *slightly *easier
readability wouldn't make up for the headache IMO.

On Sun, Mar 22, 2020 at 12:13 AM Ivan Pozdeev via Python-Dev <
python-dev@python.org> wrote:

> On 22.03.2020 6:38, Guido van Rossum wrote:
>
> On Sat, Mar 21, 2020 at 6:46 PM Nick Coghlan  wrote:
>
>> On Sat., 21 Mar. 2020, 11:19 am Nathaniel Smith,  wrote:
>>
>>> On Fri, Mar 20, 2020 at 11:54 AM Dennis Sweeney
>>>  wrote:
>>> > This is a proposal to add two new methods, ``cutprefix`` and
>>> > ``cutsuffix``, to the APIs of Python's various string objects.
>>>
>>> The names should use "start" and "end" instead of "prefix" and
>>> "suffix", to reduce the jargon factor and for consistency with
>>> startswith/endswith.
>>>
>>
>> This would also be more consistent with startswith() & endswith(). (For
>> folks querying this: the relevant domain here is "str builtin method
>> names", and we already use startswith/endswith there, not
>> hasprefix/hassuffix. The most challenging relevant audience for new str
>> builtin method *names* is also 10 year olds learning to program in school,
>> not adults reading the documentation)
>>
>
> To my language sense, hasprefix/hassuffix are horrible compared to
> startswith/endswith. If you were to talk about this kind of condition using
> English instead of Python, you wouldn't say "if x has prefix y", you'd say
> "if x starts with y". (I doubt any programming language uses hasPrefix or
> has_prefix for this, making it a strawman.)
>
> *But*, what would you say if you wanted to express the idea or removing
> something from the start or end? It's pretty verbose to say "remove y from
> the end of x", and it's not easy to translate that into a method name.
> x.removefromend(y)? Blech! And x.removeend(y) has the double 'e', which
> confuses the reader.
>
> The thing is that it's hard to translate "starts" (a verb) into a noun --
> the "start" of something is its very beginning (i.e., in Python, position
> zero), while a "prefix" is a noun that specifically describes an initial
> substring (and I'm glad we don't have to use *that* :-).
>
>
>> I think the concern about stripstart() & stripend() working with
>> substrings, while strip/lstrip/rstrip work with character sets, is valid,
>> but I also share the concern about introducing "cut" as yet another verb to
>> learn in the already wide string API.
>>
>
> It's not great, and I actually think that "stripprefix" and "stripsuffix"
> are reasonable. (I found that in Go, everything we call "strip" is called
> "Trim", and there are "TrimPrefix" and "TrimSuffix" functions that
> correspond to the PEP 616 functions.)
>
> I must note that names conforming to
> https://www.python.org/dev/peps/pep-0008/#function-and-variable-names
> would be "strip_prefix" and "strip_suffix".
>
>
>
>> The example where the new function was used instead of a questionable use
>> of replace gave me an idea, though: what if the new functions were
>> "replacestart()" and "replaceend()"?
>>
>> * uses "start" and "with" for consistency with the existing checks
>> * substring based, like the "replace" method
>> * can be combined with an extension of "replace()" to also accept a tuple
>> of old values to match and replace to allow for consistency with checking
>> for multiple prefixes or suffixes.
>>
>> We'd expect the most common case to be the empty string, but I think the
>> meaning of the following is clear, and consistent with the current practice
>> of using replace() to delete text from anywhere within the string:
>>
>> s = s.replacestart('context.' , '')
>>
>
> This feels like a hypergeneralization. In 99.9% of use cases we just need
> to remove the prefix or suffix. If you want to replace the suffix with
> something else, you can probably use string concatenation. (In the one use
> case I can think of, changing "foo.c" into "foo.o", it would make sense
> that plain "foo" ended up becoming "foo.o", so s.stripsuffix(".c") + ".o"
> actually works better there.
>
>
>> This approach would also very cleanly handle the last example from the
>> PEP:
>>
>> s = s.replaceend(('Mixin', 'Tests', 'Test'), '')
>>
>
> Maybe the proposed functions can optionally take a tuple of
> prefixes/suffixes, like startswith/endswith do?
>
>
>> The doubled 'e' in 'replaceend' isn't ideal, but if we went this way, I
>> think keeping consistency with other str method names would be preferable
>> to adding an underscore to the name.
>>
>
> Agreed on the second part, I just really don't like the 

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

[Kyle Stanley]

Nick Coghlan wrote:
> The example where the new function was used instead of a questionable use
of replace gave me an idea, though: what if the new functions were
"replacestart()" and "replaceend()"?
>
> * uses "start" and "with" for consistency with the existing checks
> * substring based, like the "replace" method
> * can be combined with an extension of "replace()" to also accept a tuple
of old values to match and replace to allow for consistency with checking
for multiple prefixes or suffixes.

FWIW, I don't place as much value on being consistent with "startswith()"
and "endswith()". But with it being substring based, I think the term
"replace" actually makes a lot more sense here compared to "cut". +1


On Sat, Mar 21, 2020 at 9:46 PM Nick Coghlan  wrote:

> On Sat., 21 Mar. 2020, 11:19 am Nathaniel Smith,  wrote:
>
>> On Fri, Mar 20, 2020 at 11:54 AM Dennis Sweeney
>>  wrote:
>> > This is a proposal to add two new methods, ``cutprefix`` and
>> > ``cutsuffix``, to the APIs of Python's various string objects.
>>
>> The names should use "start" and "end" instead of "prefix" and
>> "suffix", to reduce the jargon factor and for consistency with
>> startswith/endswith.
>>
>
> This would also be more consistent with startswith() & endswith(). (For
> folks querying this: the relevant domain here is "str builtin method
> names", and we already use startswith/endswith there, not
> hasprefix/hassuffix. The most challenging relevant audience for new str
> builtin method *names* is also 10 year olds learning to program in school,
> not adults reading the documentation)
>
> I think the concern about stripstart() & stripend() working with
> substrings, while strip/lstrip/rstrip work with character sets, is valid,
> but I also share the concern about introducing "cut" as yet another verb to
> learn in the already wide string API.
>
> The example where the new function was used instead of a questionable use
> of replace gave me an idea, though: what if the new functions were
> "replacestart()" and "replaceend()"?
>
> * uses "start" and "with" for consistency with the existing checks
> * substring based, like the "replace" method
> * can be combined with an extension of "replace()" to also accept a tuple
> of old values to match and replace to allow for consistency with checking
> for multiple prefixes or suffixes.
>
> We'd expect the most common case to be the empty string, but I think the
> meaning of the following is clear, and consistent with the current practice
> of using replace() to delete text from anywhere within the string:
>
> s = s.replacestart('context.' , '')
>
> This approach would also very cleanly handle the last example from the PEP:
>
> s = s.replaceend(('Mixin', 'Tests', 'Test'), '')
>
> The doubled 'e' in 'replaceend' isn't ideal, but if we went this way, I
> think keeping consistency with other str method names would be preferable
> to adding an underscore to the name.
>
> Interestingly, you could also use this to match multiple prefixes or
> suffixes and find out *which one* matched (since the existing methods don't
> report that):
>
> s2 = s.replaceend(suffixes, '')
> suffix_len = len(s) - len(s2)
> suffix = s[-suffix-len:] if suffix_len else None
>
> Cheers,
> Nick.
>
>
>> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VQULYFFT4VVXV35RE5ETR5MOZSHLPFTV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KFDWZ3LWUIE6KHYQYU6Z5VL3SXMMMZOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proliferation of tstate arguments.

[Kyle Stanley]

Antoine Pitrou wrote:
> In any case, the amount of disagreement and/or misunderstanding in this
> discussion is a strong hint that it needs a PEP to hash things out and
> explain them clearly, IMHO.

Agreed; a PEP (even if it's just informational) would go a long way in
helping to clear up some misunderstandings. Perhaps something along the
lines of "Supporting concurrent subinterpreters in the C-API" that provides
both a high-level overview and low-level information on some of the
different structures involved and any needed changes [1], such as to
PyThreadState, PyInterpreterState, PyGILState_STATE, etc. Optimally, it
would explain some changes made and needed changes to their public and
private APIs, as well as to the struct members.

I'd be happy to help with something like this (particularly in reviewing
and providing feedback on any unclear parts), but my own current
understanding is likely not strong enough to lead the efforts. In fact, I
very likely am in the camp of having some significant misunderstandings
about the low-level implementation details. My experience with
subinterpreters [2] is mostly limited to attempting to help with some
tricky bugs. I'd call it an "area of interest", but it's very far from an
"area of expertise". ;-)

In the meantime for anyone looking for a basic overview on thread state and
interpreter state, there's some general information documented in
https://docs.python.org/3.9/c-api/init.html#high-level-api.

---

[1] - By "needed changes", I am specifically referring to any changes to
the C-API or internals that were made or are still needed to support
concurrent subinterpreters.

[2] - I have much more experience in the existing areas of concurrency in
the standard library, such as with asyncio and concurrent.futures. Although
it's relevant, it's very different in terms to implementation details.
Also, I'm substantially more interested in the Python parts of the stdlib
rather than the C internals or extension modules, but I certainly have an
interest in learning more.

Regards,
Kyle Stanley

On Wed, Mar 18, 2020 at 11:53 AM Antoine Pitrou  wrote:

> On Wed, 18 Mar 2020 13:35:16 +
> Anders Munch  wrote:
> > Chris Angelico [mailto:ros...@gmail.com]:
> > > And by that logic, globals are even more capable. I don't understand
> your
> > > point. Isn't the purpose of the tstate parameters to avoid the problem
> of
> > > being unable to have multiple tstates within the same OS thread? I
> think I've
> > > missed something here.
> >
> > The point is that because threads can't preempt themselves, this:
> >
> > /*1*/
> > {
> > Py_something(other_tstate);
> > }
> >
> > and this:
> >
> > /*2*/
> > {
> >PyInterpreterState* old_tstate = tstate;
> >tstate = other_state;
> >Py_something();
> >tstate = old_tstate;
> > }
> >
> > is effectively equivalent, provided tstate is thread-local.  Both will
> work
> > equally well from the hypothetical C callback that wants to use a
> different
> > subinterpreter.
>
> This example is mixing up the notion of interpreter state and thread
> state.
>
> Here is a more realistic example:
>
> struct UserData {
>   PyInterpreterState* interp;
> };
>
> // Callback given to a third-party C library
> void my_c_callback(void* opaque) {
>   struct UserData* user_data = (struct UserData*) opaque;
>   PyGILState_STATE gstate;
>   gstate = PyInterpreter_GILStateEnsure(user_data->interp);
>
>   // ...
>
>   PyInterpreter_GILStateEnsure(user_data->interp, gstate);
> }
>
>
> Of course this implies the existence of a PyInterpreter_GILState API
> that currently doesn't exist.  In this model, the "thread-local" thread
> state is a *per-interpreter* thread-local.  A *process-wide*
> thread-local thread state cannot work if we want -- in some distant
> future -- to be able to run several subinterpreters concurrently.
>
>
> In any case, the amount of disagreement and/or misunderstanding in this
> discussion is a strong hint that it needs a PEP to hash things out and
> explain them clearly, IMHO.
>
> Regards
>
> Antoine.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EKZJYO75QBDKZA4A2RR43PKDYEJRD7HC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T4EC2LXRCGTIN222HNILLVKUSAOZ3BGA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proliferation of tstate arguments.

[Kyle Stanley]

Mark Shannon wrote:
> The point I'm making is that adding `tstate` parameters everywhere is
> unnecessary. Using a thread local variable is much less intrusive and is
> at least as capable.

Objectively speaking, what would be the specific difference(s) in behavior
and performance between using a thread-local variable vs passing a tstate
parameter? A more in-depth "pros vs cons" analysis of each approach might
help to make the argument points more clear, as it seems some of the
parties involved in the discussion are talking past each other to some
degree, or at least aren't completely on the same page.

It might also help to find a specific C-API function to benchmark, to show
how substantial the supposed performance differences could be. Intuitively,
it seems like passing around an extra parameter would add some penalty, but
it's not clear to me as to how much that would *realistically* add in
performance cost compared to accessing the threadstate through a
thread-local variable. It seems like it depends on exactly how often
`_ThreadState_GET()` would have to be called, but that's not at all obvious
considering the scope of the changes.

> Using a thread local variable is much less intrusive and is
> at least as capable.

I'm not sure that I'm clear on how the tstate parameter would be
additionally intrusive, considering that the changes only affect the
internal parts of the C-API. I would agree if it were to be arbitrarily
added to public API since it would break backwards compatibility, but that
doesn't seem to be the case here. From the perspective of being intrusive,
it seems like it's adding some code churn and a potential merge conflict to
resolve locally. That seems like an inconvenience to us rather than
something that would be intrusive to users.

Also, regarding the thread-local variable example demonstrated w/ godbolt
(thanks for providing those :-) ), it seems like it would be very clear to
access the *current* threadstate that was created by the *main*
interpreter, but how would you go about resolving the following?:

1) A different threadstate from another interpreter and OS thread
2) A different threadstate created from the same OS thread, but from a
different interpreter (as Antoine mentioned earlier)

>From my understanding, those scenarios seem to be a significant concern in
the context of subinterpreters. If we could also see example(s) which
address those scenarios with a thread-local variable instead of a tstate
parameter, I think it would allow for more objective comparison between
them.

Regards,
Kyle Stanley

On Wed, Mar 18, 2020 at 6:36 AM Mark Shannon  wrote:

>
>
> On 17/03/2020 7:00 pm, Steve Dower wrote:
> > On 17Mar2020 1803, Chris Angelico wrote:
> >> On Wed, Mar 18, 2020 at 3:50 AM Mark Shannon  wrote:
> >>> The accessibility of a thread-local variable is a strict superset of
> >>> that of a function-local variable.
> >>>
> >>> Therefore storing the thread state in a thread-local variable is at
> >>> least as capable as passing thread-state as a parameter.
> >>>
> >>
> >> And by that logic, globals are even more capable. I don't understand
> >> your point. Isn't the purpose of the tstate parameters to avoid the
> >> problem of being unable to have multiple tstates within the same OS
> >> thread? I think I've missed something here.
> >
> > You haven't. Separating the Python thread from the "physical" thread is
> > indeed the point.
>
> That seems like a strawman argument; maybe I'm misunderstanding Steve.
> It seems to me that Steve is implying that using thread-local variables
> somehow prevents that separation. It does not.
>
> The point I'm making is that adding `tstate` parameters everywhere is
> unnecessary. Using a thread local variable is much less intrusive and is
> at least as capable.
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/CLRCMGUC5LO3JOFGB2QG6DCJRTVVC3A4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LI7BD63VNAUTNC4JQ7VU7UZLH7F4LEOC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: New discuss.python.org category: Core Dev

[Kyle Stanley]

Thanks for adding the new section, Brett. :)

Was the name "Core Development" also considered? To me, it seems like "Core
Dev" could be interpreted as abbreviating "Core Developer", which seems
roughly equivalent to the existing "Committers" category (at least based on
the name alone). I'm sure that either would be fine as long as people read
the pinned "About this category" topic, but I suspect that the name "Core
Development" would make the purpose of the category more immediately clear.


On Tue, Mar 3, 2020 at 7:09 PM Brett Cannon  wrote:

> I just created https://discuss.python.org/c/core-dev/ to somewhat mirror
> this mailing list as people were asking for an equivalent space on
> discuss.python.org due to the Users category now being heavily used for
> user questions (which makes sense based on the title ). I also moved the
> SC Nominations to be a subcategory under Core Dev so it can still be
> publicly accessible but not hidden if people choose to mute the Users
> category (which I know has been a problem in the past for some people).
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VOYITJ3JIDJT4F5NPJE2AI2U7LGFHF7I/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OI4OD3CWXUYFPSXQSVBOSCIFTM7RSBKT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to respond to repeated bad ideas

[Kyle Stanley]

> In most cases of a first-time poster that I've seen, the poster probably
doesn't have the understanding needed to conduct a proper search of the
mailing list. That's why I suggest responding with some genuine help (i.e.
taking their idea at face value and explaining what's wrong with it).

It might also be helpful to directly link to a few of those past
discussions; particularly if they had some relevant responses that could
provide some valuable insight for the OP. I've done this a few times, and
mentioned that if they want to still proceed with their idea, they'd have
to address the previous concerns brought up. IMO, this provides a good
balance of avoiding an exact repetition of past discussions, providing the
OP with some answers, and in a best case scenario the OP might have valid
counterarguments to the previous reason(s) the proposal was rejected.

On Mon, Mar 2, 2020 at 10:12 PM Guido van Rossum  wrote:

> On Mon, Mar 2, 2020 at 2:42 PM Bar Harel  wrote:
>
>> What I usually do btw is just search on mailman. Perhaps we can guide
>> people to search on mailman before suggesting an idea?
>>
>
> We could add that suggestion to the "welcome to this list" email. I
> honestly don't know if we send out such emails, but Mailman (3) seems to
> have that capability.
>
> But I doubt that many users will follow that advice. In most cases of a
> first-time poster that I've seen, the poster probably doesn't have the
> understanding needed to conduct a proper search of the mailing list. That's
> why I suggest responding with some genuine help (i.e. taking their idea at
> face value and explaining what's wrong with it). For example, Andrew
> Barnert does this excellently in many of his python-ideas posts. Many posts
> by Steven D'Aprano are also good examples. In the past, Tim Peters was the
> champion of this style.
>
> On Mon, Mar 2, 2020 at 3:03 PM Skip Montanaro 
> wrote:
>
>> > Atm we don't have an index of ideas, apart from pep 3099, and I'm not
>> sure we can make one (can we?), so I do not see a way to prevent this from
>> happening.
>>
>> Maybe an informational PEP which briefly lists rejected ideas?
>> Presumably, they'd normally come up in python-ideas, python-list or
>> python-dev. Each rejected idea could link to one or more relevant
>> threads in one of those lists. Not sure who should be the gatemasters
>> for new bad ideas.
>>
>
> At least for python-ideas, that sounds a lot like an exhaustive list of
> all threads in that list. :-)
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/PJRWYQZSZAUB46WPNDPA3MIMHEGNRKLC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HD2FV26CLDMGSSVUYPY6QGWLXL4R6MFE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

[Kyle Stanley]

> So I've also never come across "|=" being used for this purpose.

IIRC, the JavaScript implementation of "|=" can potentially be used in the
way Claudio described it, instead it's based on the truthiness of the
left-hand operand rather than it being "unset". But it works in that
context because "null" and "undefined" are considered falsey [1]. For
example:

> var value = null;
> var other = 2;
> value |= other;
> console.log(value);
2

So it effectively works like "value | other", but also sets "value" to
"other" iff "value" is falsey. When the left-hand operand is truthy, it
effectively does nothing.

> var value = 3;
> var other = 2;
> value |= other;
> console.log(value);
3

Also worth noting, since "value |= other" translates to "value = value |
other", it works as a bitwise OR operator; not as a catch-all for assigning
a default value:

> var value = null;
> var other = "test";
> value |= other;
> console.log(value);
0

Instead, you'd have to use the standard OR operator, like this "value =
value || other" (since "||=" is invalid syntax):

> var value = null;
> var other = "test";
> value = value || other;
> console.log(value);
test

FWIW, I have very rarely seen "|=" used as an operator in JS, but I've seen
"value = value || other" used a decent amount.

---

[1] - https://developer.mozilla.org/en-US/docs/Glossary/Falsy



On Wed, Feb 26, 2020 at 6:26 PM Nick Coghlan  wrote:

>
>
> On Thu., 27 Feb. 2020, 2:03 am Guido van Rossum,  wrote:
>
>> On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz 
>> wrote:
>>
>>> In my experience, the expression `value |= other` is a common idiom
>>> across
>>> programming languages to provide a default for `value` if it is "unset".
>>>
>>
>> Interesting. Can you point to specific examples of this? In what other
>> languages have you seen this? (Not that it would make us change PEP 584,
>> but if this appears common we could probably warn about it prominently in
>> docs and tutorials.)
>>
>
> I was thinking that bash scripting might be an example, but I double
> checked, and that's spelled 'VAR="${$VAR:-default value}" '
>
> make has 'VAR ?= "default value"'
>
> C# uses "??=" for null coalescence on assignment.
>
> So I've also never come across "|=" being used for this purpose.
>
> Cheers,
> Nick.
>
>> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HMKYUZP5T6HTURG46GU3L72KANB65MLQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FOL22B6MXU65P2T3M6CCMYVZSSSWR2PO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Merging PRs without CLA signed

[Kyle Stanley]

> I remember seeing it too.  It may have originally been in the tracker
> instructions, but should definitely be in the devguide now.

>From looking through the devguide for every instance of "CLA" and
"trivial", there seems to be just one section that mentions anything
regarding the triviality of the patch in relation to the CLA:

"If the submitter lacks a signed CLA and the patch is non-trivial, direct
them to the electronic Contributor Licensing Agreement
<https://www.python.org/psf/contrib/contrib-form/> to ensure the PSF has
the appropriate authorizations in place to relicense and redistribute their
code."

(https://devguide.python.org/committing/#handling-others-code)

Since the licensing section of the pullrequest page is linked to from the
index of the devguide, as being the main reference for first-time
contributors signing the CLA, I think we could benefit from mentioning it
there as well:

"To accept your change we must have your formal approval for distributing
your work under the PSF license
<https://docs.python.org/dev/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python>.
Therefore, you need to sign a contributor agreement
<https://www.python.org/psf/contrib/> which allows the Python Software
Foundation <https://www.python.org/psf/> to license your code for use with
Python (you retain the copyright)."

(https://devguide.python.org/pullrequest/#cla)

This could involve a very minimal change to the first sentence, which would
help both sections to better convey the same policy [1]:

"To accept any non-trivial change we must have..."
Would that be reasonable? It wouldn't change the existing policy (since
it's mentioned on the committing page), but I think it would make it more
visibly clear that the CLA isn't a hard requirement for trivial PRs. This
would likely just reinforce the current state of affairs, where it's
ultimately at the discretion of the core dev(s) reviewing the patch [2];
while hopefully clearing up misconceptions.

I think it would also be useful to briefly describe guidelines on the
committing page that help to define what may constitute as trivial vs
non-trivial, along the lines of how Terry described the distinction. I
personally found the comparison of minor fact correction vs creative
expression to be particularly helpful. But, a section like that would
likely require some oversight from the PSF legal staff.

---

[1] - From my interpretation at least, "To accept your change we must have
..." and "If the submitter lacks a signed CLA and the patch is non-trivial
..." seem to be mildly in conflict of one another regarding the CLA policy.

[2] - Discretion to determine triviality based on best judgement, and
whether or not they personally consider merging any PRs without the CLA
signed.

On Mon, Feb 24, 2020 at 1:54 PM Terry Reedy  wrote:

> On 2/23/2020 11:44 PM, Guido van Rossum wrote:
> > On Sun, Feb 23, 2020 at 8:11 PM Kyle Stanley  > <mailto:aeros...@gmail.com>> wrote:
> >
> > In a recently opened typo fixing PR [1], an issue came up regarding
> > the lack of a signed CLA, where the author specifically mentioned
> > they did not want to sign it for privacy concerns.
> >
> >
> > In that case I'm not sure the author ought to get credit for the PR.
>
> If the account has a real name, then there cannot be a privacy concern.
> If if does not, then credit can only be claimed privately.
>
> > They can file a bug pointing out the typo and someone else can submit a
> > fix.
>
> One of that justifications given for moving to github was that is would
> allow trivial changes to be submitted without an issue.  Allowing merges
> for trivial changes without a CLA was intentional, after discussion.
>
> To summarize my response a few minutes ago to Antoine and Chris
> Angelico, I consider trivial to mean non-copyrightable because short and
> fact based.
>
> > (This is what Glyph had to do for all his contributions while he
> > was employed at Apple.)
>
> And what anyone in a similar situation should still do for anything
> non-trivial.
> > Yeah, typically we don't insist on a CLA for trivial fixes -- it's at
> > the discretion of the core dev reviewing/merging the PR. I actually
> > thought this was a policy that was written down somewhere, but I don't
> > know where (maybe somewhere in the devguide?).
>
> I remember seeing it too.  It may have originally been in the tracker
> instructions, but should definitely be in the devguide now.
>
>
>
>
> --
> Terry Jan Reedy
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailma

[Python-Dev] Re: Merging PRs without CLA signed

[Kyle Stanley]

> In that case I'm not sure the author ought to get credit for the PR. They
can file a bug pointing out the typo and someone else can submit a fix.

That sounds like a reasonable solution to me; even for more substantial
issues (if signing the CLA is a genuine issue). I think there are a fair
number of individuals out there who just want to fix something and aren't
concerned with attributions or long-term contributions; they just want to
fix the issue for themselves or perhaps for altruistic reasons.

> In the past the people who refused to sign the CLA just had some beef
with the legal system -- that's fine, it's their choice, but we just cannot
accept their contributions: that's our choice.

Yeah, I think that's fair.

> (What is it with typos anyway? Why do people feel the need to invoke
megabytes if not gigabytes of internet traffic to correct a word that every
reader can easily correct in their mind?)

Speaking from personal experience to some degree, my first PR was an
incredibly minimal documentation enhancement:
https://github.com/python/cpython/pull/14069. It's not exactly a typo fix,
but in retrospect, I'd consider it to be about equally impactful. I can't
speak for everyone, but my own motivation was to do something very mildly
helpful just to get a feel for the workflow. It eventually led to
increasingly involved contributions. (:

I think some people might also consider grammatical corrections to be
helpful for bolstering the "professionalism" of the documentation a bit,
but it's hard to say for sure.

> Honestly it seems a rather trivial matter to be so concerned about
fairness. Hopefully a contributor isn't really going to claim "Python Core
Contributor" on their resume based on a typo fix they contributed, and if
they do, I'm not sure whether the CLA requirement is really the key issue
of fairness...

Haha, I had honestly not considered that perspective. But yeah, hopefully
in that case the potential employer would look into the actual contribution
or ask them to link their GitHub, rather than just taking it at face
value...

My concern was mostly just that it might turn some first-contributors away
if they open their first minimal PR, are required to sign the CLA, and then
see that others for similar (or even more involved) PRs didn't have to sign
it. It also has came up enough times that I'd like to have a clear answer
to provide. Perhaps "fairness" could be overstating the issue though.

I was also wondering if there might be any licensing issues for having a
decent volume of total contributions to CPython from individuals without a
CLA signed (even if they're minimal by themselves), but that's likely a
question for the PSF legal team rather than python-dev.

> Write down explicitly that for truly trivial PRs it's at the discretion
of the reviewer?

IMO, that would still be an improvement, because at least then everyone
would have a definitive policy to refer to, even if that policy is "it's up
to the core dev reviewing the PR". I would very much like to know whether
signing the CLA for all PRs is intended to be a concrete policy or if it's
at the discretion of those reviewing the PR. Over the course of my PR
reviews, I've seen quite a number of mixed answers. It's not an absurdly
common issue that is asked on every other PR, but I think it comes up
enough to justify having a more clearly defined policy of some form.

> I believe I've heard that the FSF has a similar policy that states a
maximum number of lines or characters for PRs to be considered possibly
trivial -- but since it's sometimes possible to contribute a truly amazing
speedup that's only a few characters in size, it really ought to be up to
the core dev. Or maybe it should be limited to at most a handful of typo,
grammar or punctuation fixes in docs or comments. (And no splitting it up
into a multiple PRs to duck the limit.)

Both of those also sounds reasonable, and would be aligned with the
existing similar PRs that have been merged without a signed CLA (as far as
I can tell). As mentioned in the OP, I don't have an especially strong
opinion on how it should be handled. More than anything, I'd just like the
policy to be made clear for future PRs so that I can provide an accurate
answer to newer contributors.

On Sun, Feb 23, 2020 at 11:44 PM Guido van Rossum  wrote:

> On Sun, Feb 23, 2020 at 8:11 PM Kyle Stanley  wrote:
>
>> In a recently opened typo fixing PR [1], an issue came up regarding the
>> lack of a signed CLA, where the author specifically mentioned they did not
>> want to sign it for privacy concerns.
>>
>
> In that case I'm not sure the author ought to get credit for the PR. They
> can file a bug pointing out the typo and someone else can submit a fix.
> (This is what Glyph had to do for all his contributions while he was
> employed at Apple.)
>
> While it's *possible* that there are authors th

[Python-Dev] Re: Merging PRs without CLA signed

[Kyle Stanley]

> Note that if you open a PR, and _then_ sign the CLA, the label is not
updated (at least, that's what I experienced before I did). So this list is
likely inaccurate.

I believe that I might have seen this happen a few times, but in the
majority cases the label is updated from "CLA not signed" => "CLA signed".
There's a bit of delay (~24-48 hours) from the time you sign the CLA to
when it's updated in the heroku app, so if the PR gets merged during that
time the label might not ever update.

For my own first PR, it was initially not signed when I opened it, and then
the label updated to signed within a couple days after signing the CLA.
This has also been the case in the majority of PRs I've reviewed from
first-time contributors.

On Sun, Feb 23, 2020 at 11:24 PM Ivan Pozdeev via Python-Dev <
python-dev@python.org> wrote:

>
> On 24.02.2020 7:07, Kyle Stanley wrote:
> >
> > For a full list of merged PRs to CPython with a "CLA not signed" label,
> see the following:
> >
> https://github.com/python/cpython/pulls?utf8=%E2%9C%93=is%3Apr+state%3Amerged+label%3A%22CLA+not+signed%22
> >
> Note that if you open a PR, and _then_ sign the CLA, the label is not
> updated (at least, that's what I experienced before I did). So this
> list is likely inaccurate.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IO66YO2SVW55M5KXCAM2P4J7DVMSCK5X/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F22BOEMMBFZJMVKGVR7YDTR5ZXJKIXAY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Merging PRs without CLA signed

[Kyle Stanley]

In a recently opened typo fixing PR [1], an issue came up regarding the
lack of a signed CLA, where the author specifically mentioned they did not
want to sign it for privacy concerns.

In the past, I've seen several PRs with similarly minimal [2] changes (such
as typo fixes, grammar fixes, link fixes, etc) merged without having the
CLA signed, so it was my assumption that this was acceptable. For a full
list of merged PRs to CPython with a "CLA not signed" label, see the
following:
https://github.com/python/cpython/pulls?utf8=%E2%9C%93=is%3Apr+state%3Amerged+label%3A%22CLA+not+signed%22

However, I was informed by Pablo Galindo that there are legal issues
involved with merging *any* PRs without the CLA signed, including typo
fixes. Personally, I have no strong opinion one way or the other, as I
don't have an adequate understanding from a legal/licensing perspective.
But, I think think there's definitely an issue with the lack of consistency
regarding this policy.

*To require a signed CLA for some minimal PRs but not others, solely based
on who happens to be reviewing the PR, seems rather unfair to potential
contributors.* From my perspective, the solution seems to be clearly
defining a more explicit stance on this policy, and having it apply as
universally as possible to *all* PRs made to CPython.

For example, if the CLA should be required for all PRs, the policy might
state something like this: "The author of any PR made to the CPython
repository must have signed the CLA before their PR can be merged. Any PR
opened by an author that has not signed the CLA can't be merged until it
has been signed."

OTOH, if it's okay for minimal PRs to not have a signed CLA: "The author of
any PR made to the CPython repository must have signed the CLA before their
PR can be merged, except for minimal PRs. Some examples of minimal PRs
include: ..."

Currently, the policy seems to be learning more towards the former based on
the devguide [3], where it states "To accept your change we must have your
formal approval for distributing your work under the PSF license.
Therefore, you need to sign a contributor agreement which allows the Python
Software Foundation to license your code for use with Python (you retain
the copyright)".

However, it seems apparent to me that either this policy isn't explicit
enough, has a lack of visibility, or simply isn't followed consistently.
What might be a viable solution to this problem?

---

[1] - https://github.com/python/cpython/pull/18603

[2] - The term "minimal" can be interchanged with "trivial" for the most
part in the above context, but I tend to prefer the former. IMO, it comes
across as more respectful to the efforts made by the author, as even the
smallest of PRs can require substantial efforts from first-time
contributors that are entirely unfamiliar with the workflow; regardless of
how small the change is.

[3] - https://devguide.python.org/pullrequest/#licensing

Regards,
Kyle Stanley
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LH6WQI7GGS6URLXOGYGAPJIOXKGVIX2Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Three trivial PRs from first-timers in need of merging!

[Kyle Stanley]

Thanks for bring attention to these PRs, Brandt! I think the second one
should be particularly uncontroversial, seeing as it's just applying PEP
409 (raise from None) to an existing exception in an argparse unit test to
clean up some unhelpful context clutter in the traceback.

On Thu, Feb 20, 2020 at 11:41 PM Brandt Bucher 
wrote:

> Hi everybody!
>
> I've found a handful of trivial PRs from new contributors that have gone
> un-core-reviewed since November. CLAs are signed, tests are passing, and
> I've verified the quality and correctness of each one.
>
> If anybody has a few spare minutes to merge these, I know that it would
> make the authors very happy! :)
>
> Clarify numeric padding behaviour in string formatting:
> - A helpful doc addition (just a few words).
> - https://github.com/python/cpython/pull/17036
>
> argparse unittest tracebacks are confusing if an error is raised when not
> expected:
> - This just changes a "raise" to a "raise from None" in the argparse unit
> test machinery.
> - https://github.com/python/cpython/pull/17120
>
> Reuse identifier of PREDICT macros as PREDICT_ID:
> - Unifies some shared naming logic in the ceval prediction macros (don't
> worry, it's simple).
> - https://github.com/python/cpython/pull/17155/files
>
> Thanks.
>
> Brandt
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/46U2H5U2BAACZZETIWJTAJAGXJSD6MKO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PAOV4YWV4FJDM7WEOJ2DSMCO4XWHWFWZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Regarding SSL installation in python

[Kyle Stanley]

Hey Chaitanya,

The "python-dev" mailing list is specifically for the development *of*
Python itself, not for general help with Python. "comp.lang.python" or
"python-help" is more likely the list you're looking for.

Also, for general information on the purpose of each of the primary mailing
lists, see https://www.python.org/community/lists/.

On Mon, Feb 17, 2020 at 1:04 AM Naga Chaitanya L <
nagachaitany...@vasudhaika.net> wrote:

> Hi,
> I want to know how to install ssl certificate in python. please give me
> referral link.
> thnk u
> Regards!
> chaitanya
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LS6OFKHCJILJIYRHRDCTHEMUHSAU47CE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D4OPKMFXQDIZSZBQKCYONRURTWCDYT5L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Azure Pipelines PR: Unresponsive agent

[Kyle Stanley]

 > I think we're at the point where it's probably okay to disable Azure
> Pipelines as a required check and replace it with the GitHub Actions
checks.

Sounds good, GitHub Actions CI looks like it's been working smoothly. Since
you introduced it, I haven't encountered any issues in my own PRs or ones
that I've reviewed.

> But ignoring that for now, I think it's probably best to re-run CI
> (close/reopen). Just in case the change did actually cause a problem
> that may only show up on that particular configuration. The agent isn't
> actually within our control, so it'll be recreated automatically.

Thanks for the advice! I figured this would be the best option in this
situation, but since I wasn't sure about how exactly the agents worked, it
seemed like a good idea to ask first.



On Sat, Feb 1, 2020 at 6:27 AM Steve Dower  wrote:

> On 01Feb2020 1840, Kyle Stanley wrote:
> > In a recent PR (https://github.com/python/cpython/pull/18057), I
> > received the following error message in the Azure Pipelines build
> results:
> >
> > ##[error]We stopped hearing from agent Azure Pipelines 5. Verify the
> > agent machine is running and has a healthy network connection. Anything
> > that terminates an agent process, starves it for CPU, or blocks its
> > network access can cause this error. For more information, see:
> > https://go.microsoft.com/fwlink/?linkid=846610
> >
> > Build:
> >
> https://dev.azure.com/Python/cpython/_build/results?buildId=57319=results
> >
> > Is there something on our end we can do to bring the agent back online,
> > or should I simply wait a while and then try to restart the PR checks?
> > Normally I'd avoid doing that, but in this case it's entirely unrelated
> > to the PR.
>
> I think we're at the point where it's probably okay to disable Azure
> Pipelines as a required check and replace it with the GitHub Actions
> checks.
>
> But ignoring that for now, I think it's probably best to re-run CI
> (close/reopen). Just in case the change did actually cause a problem
> that may only show up on that particular configuration. The agent isn't
> actually within our control, so it'll be recreated automatically.
>
> (FWIW, the two failing buildbots on the PR are unsupported for 3.9, but
> haven't been disabled yet.)
>
> Cheers,
> Steve
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WANEWGWPZKN2KZGRWSC55DEF7RHRMQ4X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Azure Pipelines PR: Unresponsive agent

[Kyle Stanley]

In a recent PR (https://github.com/python/cpython/pull/18057), I received
the following error message in the Azure Pipelines build results:

##[error]We stopped hearing from agent Azure Pipelines 5. Verify the agent
machine is running and has a healthy network connection. Anything that
terminates an agent process, starves it for CPU, or blocks its network
access can cause this error. For more information, see:
https://go.microsoft.com/fwlink/?linkid=846610

Build:
https://dev.azure.com/Python/cpython/_build/results?buildId=57319=results

Is there something on our end we can do to bring the agent back online, or
should I simply wait a while and then try to restart the PR checks?
Normally I'd avoid doing that, but in this case it's entirely unrelated to
the PR.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2NSPJUEWULTLLALR3HY3H2PRYAUT474C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python Language Summit at PyCon 2020

[Kyle Stanley]

On the attendance application, there is currently an incorrect link to the
informational page for the Language Summit: "
https://us.pycon.org/2020/events/language-summit/;.

Either the link could be changed to "
https://us.pycon.org/2020/events/languagesummit/; or "
https://us.pycon.org/2020/events/language-summit/; could be configured to
redirect towards the correct page.

On Tue, Jan 28, 2020 at 4:17 PM Mariatta  wrote:

> (cross posting to python-committers and python-dev)
>
> I'm happy to announce that the signups for Python Language Summit at PyCon
> 2020 is now open.
>
> Full details at: https://us.pycon.org/2020/events/languagesummit/
>
> *TL;DR*
>
> When: Wednesday, April 15, 2020, 9am–4pm (Note, we’re starting 1 hour
> earlier than usual!)
> Where: David L. Lawrence Convention Center, Pittsburgh, PA, room TBD
>
> Sign up to attend: https://forms.gle/Fg7ayhYTaY75J1r7A (closes Feb 29th,
> 2020 AoE)
> Sign up to discuss a topic: https://forms.gle/g4BXezH1Vcn7tLds5 (closes
> Feb 29th, 2020 AoE)
>
> *Who can attend*
>
> We welcome Python core developers, active core contributors to Python and
> alternative Python implementations, and anyone else who has a topic to
> discuss with core developers.
>
> *Who can propose a discussion topic*
>
> If you have discussion items; seeking consensus; awaiting decision on a
> PEP; needing help with your core dev work; or have specific questions that
> need answers from core developers, please submit a proposal. According to
> last year’s feedback, our audience prefer more discussions and shorter
> talks.
>
> To get an idea of past language summits, you can read past years' coverage:
> 2019:
> http://pyfound.blogspot.com/2019/05/the-2019-python-language-summit.html
> 2018: https://lwn.net/Articles/754152/
> 2017: https://lwn.net/Articles/723251/
>
> This year's event will be covered by A. Jesse Jiryu Davis again, and will
> be posted on PSF's blog.
> 
>
> Some changes to note this year:
> 1) We plan to start 1 hour earlier (9AM)
> 2) The room will have U-shaped table layout
>
> Thanks!
>
>
> Mariatta & Łukasz
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/PUFQESFTN5ZUX64MKOO76LWS7W2N33M7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WCZIBRCUEKPFGPRLDTS4V23BCRB4Z6A2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Kyle Stanley]

> The only `OrderedSet` use I have seen in the wild is
https://github.com/python-hyper/uritemplate/search?q=orderedset_q=orderedset
.

A more generalized Python code search across GitHub of "orderedset" returns
~500k results: https://github.com/search?l=Python=orderedset=Code .

Of course, a general code search across GitHub is by no means a clear or
reliable measure of actual usage, considering the number of duplicate
results. But, it's still useful for rough estimations.

On Thu, Jan 2, 2020 at 7:08 PM Brett Cannon  wrote:

> The only `OrderedSet` use I have seen in the wild is
> https://github.com/python-hyper/uritemplate/search?q=orderedset_q=orderedset
> .
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GYIOTCKMKAYK2T6AALEDLZQVWCWLMXQD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/W2SY6WM5QT6TZUBRUY7Y7TFIOMFUKY7S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Kyle Stanley]

Tim Peters wrote:
> Sorry!  A previous attempt to reply got sent before I typed anything :-(

No worries, I only saw the link in the footer to the PSF CoC, and I was
mildly concerned for a moment. My first thought was "Oh no, what did I
do?". Thanks for clearing that up (:

> The collision resolution strategy for sets evolved to be fancier than
> for dicts, to reduce cache misses.  This is important for sets because
> the _only_ interesting thing about an element wrt a set is whether or
> not the set contains it.   Lookup speed is everything for sets.

Huh, that's quite interesting. For some reason, I had assumed in the back
of my head (without giving it much thought) that the average collision rate
would be the same for set items and dict keys. Thanks for the useful
information.

> If you use a contiguous range of "reasonable" integers for keys, the
> integer hash function is perfect:  there's never a collision.  So any
> such test misses all the work Raymond did to speed set lookups.
> String keys have sufficiently "random" hashes to reliably create
> collisions, though.  Cache misses can, of course, have massive effects
> on timing.

Ah, I forgot to consider how the hash function actually works for
continuous integers. A better comparison to demonstrate the collision
differences would likely use random strings.

Also, I believe that max "reasonable" integer range of no collision is
(-2305843009213693951, 2305843009213693951), as defined in Include/pyhash.h
(
https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Include/pyhash.h#L28
).

> In the former case you're primarily measuring the time to look up the
> "add" method of sets (that's more expensive than adding 0 to the set).
> A better comparison would, e.g., move `add = s.add` to a setup line,
> and use plain "add(0)" in the loop.

Good point, there's also what Chris Angelico pointed out as well: dicts
have a significant advantage in this case for having a literal for
initialization. From my understanding, this ends up having a minimal
performance impact in most realistic code (similar to method lookup time),
but it makes a very substantial difference in this case since
initialization of the containers takes up a significant portion of each
iteration.

I suspect my initialization comparison might also be flawed for similar
reasons, but I'm glad that at least 3/5 of my comparisons were mostly
reasonable. So far, the performance difference between set.update() and
dict.update() were the most notable.

I'll likely spend some more time experimenting in the next couple of weeks
or so, and report back if I find any interesting results. In the meantime,
anyone can certainly feel free to improve upon my existing comparisons.


On Mon, Dec 23, 2019 at 11:03 PM Tim Peters  wrote:

> Sorry!  A previous attempt to reply got sent before I typed anything :-(
>
> Very briefly:
>
> > >>> timeit.timeit("set(i for i in range(1000))", number=100_000)
>
> [and other examples using a range of integers]
>
> The collision resolution strategy for sets evolved to be fancier than
> for dicts, to reduce cache misses.  This is important for sets because
> the _only_ interesting thing about an element wrt a set is whether or
> not the set contains it.   Lookup speed is everything for sets.
>
> If you use a contiguous range of "reasonable" integers for keys, the
> integer hash function is perfect:  there's never a collision.  So any
> such test misses all the work Raymond did to speed set lookups.
> String keys have sufficiently "random" hashes to reliably create
> collisions, though.  Cache misses can, of course, have massive effects
> on timing.
>
> > Add (much faster for dicts):
> > >>> timeit.timeit("s = set(); s.add(0)", number=100_000_000)
> > 13.330938750001224
> > >>> timeit.timeit("d = {}; d[0] = None", number=100_000_000)
> > 5.788865385999088
>
> In the former case you're primarily measuring the time to look up the
> "add" method of sets (that's more expensive than adding 0 to the set).
> A better comparison would, e.g., move `add = s.add` to a setup line,
> and use plain "add(0)" in the loop.
>
> That's it!
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LDAC4526CZZWFMXORSOKPW2PM6HZE252/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Kyle Stanley]

Chris Angelico wrote:
> I think this is an artifact of Python not having an empty set literal.
> [snip]
> When both use the constructor call or both use a literal, the
> difference is far smaller. I'd call this one a wash.

Ah, good catch. I hadn't considered that it would make a substantial
difference, but that makes sense. Here's an additional comparison between
"{}"  and "dict()" to confirm it:

>>> timeit.timeit("{}", number=100_000_000)
2.1038335599987477
>>> timeit.timeit("dict()", number=100_000_000)
10.22558353268

Some more in-depth comparisons might be required for addition of single
items to the containers. I might experiment further with this at some point
in the next week or so, likely with implementing proper tests that omit the
time to initialize the container.








On Mon, Dec 23, 2019 at 10:09 PM Chris Angelico  wrote:

> On Tue, Dec 24, 2019 at 1:57 PM Kyle Stanley  wrote:
> > Add (much faster for dicts):
> > >>> timeit.timeit("s = set(); s.add(0)", number=100_000_000)
> > 13.330938750001224
> > >>> timeit.timeit("d = {}; d[0] = None", number=100_000_000)
> > 5.788865385999088
>
> I think this is an artifact of Python not having an empty set literal.
>
> >>> timeit.timeit("s = set(); s.add(0)", number=100_000_000)
> 13.275540543720126
> >>> timeit.timeit("d = dict(); d[0] = None", number=100_000_000)
> 13.044076398015022
> >>> timeit.timeit("d = {}; d[0] = None", number=100_000_000)
> 6.088695731014013
> >>> timeit.timeit("s = {1}; s.add(0)", number=100_000_000)
> 9.260965215042233
> >>> timeit.timeit("d = {1:2}; d[0] = None", number=100_000_000)
> 8.75433829985559
>
> When both use the constructor call or both use a literal, the
> difference is far smaller. I'd call this one a wash.
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ODZYHNI57MFZD3I7TGP3B3HJTRX36KGB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/COUO6PFHUADHYP5KSHMPHPIUCOAXS56L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should set objects maintain insertion order too?

[Kyle Stanley]

> To begin with, please compare performance of dict-with-None-values to
that of a set, for operations that can be expressed by both (e.g. both have
update()).

Good idea. It would be useful to have a baseline comparison. I emboldened
the comparisons with a notable difference.

Tested on:
Version: Python 3.8.0
OS: Linux kernel 5.4.5
CPU: Intel i5-4460

Initialization (about the same):
>>> timeit.timeit("set(i for i in range(1000))", number=100_000)
4.878508095000143
>>> timeit.timeit("{i: None for i in range(1000)}", number=100_000)
4.9192055170024105

Membership (about the same):
>>> timeit.timeit("random.randint(0, 999) in s", setup="import random; s =
set(i for i in range(1000))", number=10_000_000)
7.992674231001729
>>> timeit.timeit("random.randint(0, 999) in d", setup="import random; d =
{i: None for i in range(1000)}", number=10_000_000)
8.032404395999038

*Add* (much faster for dicts):
>>> timeit.timeit("s = set(); s.add(0)", number=100_000_000)
13.330938750001224
>>> timeit.timeit("d = {}; d[0] = None", number=100_000_000)
5.788865385999088

*Update* (much faster for updating sets):
>>> timeit.timeit("s.update(other_s)", setup="s = set(); other_s = set(i
for i in range(1000))", number=1_000_000)
6.2428832090008655
>>> timeit.timeit("d.update(other_d)", setup="d = {}; other_d = {i: None
for i in range(1000)}", number=1_000_000)
13.031371458000649

*Create list from keys* (faster for sets):
>>> timeit.timeit("list(s)", setup="s = set(i for i in range(1000))",
number=10_00_000)
5.24364303600305
>>> timeit.timeit("list(d)", setup="d = {i: None for i in range(1000)}",
number=10_00_000)
6.303017043999716

Removal isn't easily comparable, since s.pop(), s.discard(), d.pop(), and
d.popitem() all behave quite differently. Also, I'm sure these benchmarks
could be improved significantly, particularly with the "Add" ones. This
should provide a decent general idea though.

On Mon, Dec 23, 2019 at 8:12 PM Guido van Rossum  wrote:

> To begin with, please compare performance of dict-with-None-values to that
> of a set, for operations that can be expressed by both (e.g. both have
> update()).
>
> On Mon, Dec 23, 2019 at 18:08 Kyle Stanley  wrote:
>
>> Larry Hastings wrote:
>> > a hypothetical collections.OrderedSet would probably work just fine.
>>
>> I'm also in agreement with adding a collections.OrderedSet. There
>> certainly seems to be a practical use case for a Set that preserves
>> insertion order, but with significant push back to modifying the existing
>> Set implementation (primarily due to performance concerns).
>>
>> If later down the road an improvement to OrderedSet is made that gives it
>> equal or better average performance across the board compared to Set, we
>> can consider changing the default implementation *if* there's
>> significant demand for it.
>>
>> Larry Hastings wrote:
>> > And he'd probably use it too, as that makes the code clearer / easier
>> to read.  If you read code using an OrderedSet, you know what the intent
>> was and what the semantics are.  If you see code using a dict but setting
>> the values to None, you think "that's crazy" and now you have a puzzle to
>> solve.  Let's hope those comments are accurate!
>>
>> This is an excellent point. My first thought if someone were to be using
>> a Dict with all values set to None would be: why not use a Set here? As
>> Larry said, the need for preservation of insertion order would have to be
>> explicitly described in the code comments. Realistically speaking, code
>> comments are not something that can be consistently relied upon, especially
>> if it's part of some boilerplate format that gets replicated or if the
>> container is used in multiple locations. I'd much rather have a dedicated
>> data structure that clearly describes what it does based on the name alone,
>> IMO that's a million times better for readability purposes.
>>
>> Also, this is mostly speculation since I haven't ran any benchmarks for
>> an OrderedSet implementation, but I strongly suspect that OrderedSet will
>> end up having better average performance for add, pop, and update than
>> trying to use a dictionary with None values (assuming it's implemented in C
>> or at least with a C accelerator).
>>
>> Not to mention allowing the ability to update (for both addition and
>> removal) based on intersections and unions across ordered sets, which
>> currently isn't possible to do with dictionaries (at least not directly
>> with |=, &a

[Python-Dev] Re: Should set objects maintain insertion order too?

[Kyle Stanley]

Larry Hastings wrote:
> a hypothetical collections.OrderedSet would probably work just fine.

I'm also in agreement with adding a collections.OrderedSet. There certainly
seems to be a practical use case for a Set that preserves insertion order,
but with significant push back to modifying the existing Set implementation
(primarily due to performance concerns).

If later down the road an improvement to OrderedSet is made that gives it
equal or better average performance across the board compared to Set, we
can consider changing the default implementation *if* there's significant
demand for it.

Larry Hastings wrote:
> And he'd probably use it too, as that makes the code clearer / easier to
read.  If you read code using an OrderedSet, you know what the intent was
and what the semantics are.  If you see code using a dict but setting the
values to None, you think "that's crazy" and now you have a puzzle to
solve.  Let's hope those comments are accurate!

This is an excellent point. My first thought if someone were to be using a
Dict with all values set to None would be: why not use a Set here? As Larry
said, the need for preservation of insertion order would have to be
explicitly described in the code comments. Realistically speaking, code
comments are not something that can be consistently relied upon, especially
if it's part of some boilerplate format that gets replicated or if the
container is used in multiple locations. I'd much rather have a dedicated
data structure that clearly describes what it does based on the name alone,
IMO that's a million times better for readability purposes.

Also, this is mostly speculation since I haven't ran any benchmarks for an
OrderedSet implementation, but I strongly suspect that OrderedSet will end
up having better average performance for add, pop, and update than trying
to use a dictionary with None values (assuming it's implemented in C or at
least with a C accelerator).

Not to mention allowing the ability to update (for both addition and
removal) based on intersections and unions across ordered sets, which
currently isn't possible to do with dictionaries (at least not directly
with |=, &=, -=. and ^=). I'm not sure how much of a practical use case
this would have for ordered sets specifically, but it's a nice bonus.

On Sat, Dec 21, 2019 at 7:59 AM Larry Hastings  wrote:

>
> (mixing together messages in the thread, sorry threaded-view readers)
>
> On 12/19/19 3:15 PM, Tim Peters wrote:
>
> [Nick]
>
> I took Larry's request a slightly different way:
>
> [...]
>
> Only Larry can answer whether that would meet his perceived need.  My
> _guess_ is that he wouldn't know OrderedSet existed, and, even if he
> did, he'd use a dict with None values anyway (because it's less hassle
> and does everything he wanted).
>
>
> At last, something I can speak knowledgeably about: Larry's use case!
> Regarding Larry, I'd say
>
>1. his use case was small enough that almost anything maintaining
>order would have worked, even a list,
>2. he often *does have* a pretty good idea what goodies are salted
>away in the Python standard library, and
>3. a hypothetical collections.OrderedSet would probably work just
>fine.  And he'd probably use it too, as that makes the code clearer /
>easier to read.  If you read code using an OrderedSet, you know what the
>intent was and what the semantics are.  If you see code using a dict but
>setting the values to None, you think "that's crazy" and now you have a
>puzzle to solve.  Let's hope those comments are accurate!
>
>
> Also, speaking personally, at least once (maybe twice?) in this thread
> folks have asked "would YOU, Mr Larry, really want ordered sets if it meant
> sets were slightly slower?"
>
> The first thing I'd say is, I'm not sure why people should care about
> what's best for me.  That's sweet of you!  But you really shouldn't.
>
> The second thing is, my needs are modest, so the speed hit we're talking
> about would likely be statistically insignificant, for me.
>
> And the third thing is, I don't really put the set() API through much of a
> workout anyway.  I build sets, I add and remove items, I iterate over them,
> I do the occasional union, and only very rarely do I use anything else.
> Even then, when I write code where I reach for a fancy operation like
> intersection or symmetric_difference--what tends to happen is, I have a
> rethink and a refactor, and after I simplify my code I don't need the fancy
> operations anymore.  I can't remember the last time I used one of these
> fancy operations where the code really stuck around for a long time.
>
> So, personally?  Sure, I'd take that tradeoff.  As already established, I
> like that dict objects maintain their order, and I think it'd be swell if
> sets maintained their order too.  I suspect the performance hit wouldn't
> affect *me* in any meaningful way, and I could benefit from the
> order-maintaining semantics.  I bet it'd have other 

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Kyle Stanley wrote:
> or making *object* a positional only argument.

Typo: I meant "positional only parameter", not "argument".

On Mon, Dec 16, 2019 at 4:39 AM Kyle Stanley  wrote:

>
> Inada Naoki wrote:
> > If we find it broke some software, we can step back to regular
> > deprecation workflow.
> > Python 3.9 is still far from beta yet.  That's why I'm +1 on these
> proposals.
>
> IMO, since this would be changing a builtin function, we should at least
> use a version+2 deprecation cycle (in this case, removal in 3.11)
> regardless of reported breakages.
>
> Especially if there's no _substantial_ security, efficiency, or
> performance reason for immediate prevention of str() without passing an
> object (while specifying *encoding* and/or *error)  or making *object* a
> positional only argument.
>
> On Mon, Dec 16, 2019 at 4:31 AM Inada Naoki 
> wrote:
>
>> On Mon, Dec 16, 2019 at 6:25 PM Inada Naoki 
>> wrote:
>> >
>> > +1 for 1 and 2.
>> >
>>
>> If we find it broke some software, we can step back to regular
>> deprecation workflow.
>> Python 3.9 is still far from beta yet.  That's why I'm +1 on these
>> proposals.
>>
>> --
>> Inada Naoki  
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/HWNLBBHSVB5NRQC6ESQQNCQQ2EYUMW27/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAQZGR37QU6BS6UVSW4H7F4MDOYFY5ZG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Inada Naoki wrote:
> If we find it broke some software, we can step back to regular
> deprecation workflow.
> Python 3.9 is still far from beta yet.  That's why I'm +1 on these
proposals.

IMO, since this would be changing a builtin function, we should at least
use a version+2 deprecation cycle (in this case, removal in 3.11)
regardless of reported breakages.

Especially if there's no _substantial_ security, efficiency, or performance
reason for immediate prevention of str() without passing an object (while
specifying *encoding* and/or *error)  or making *object* a positional only
argument.

On Mon, Dec 16, 2019 at 4:31 AM Inada Naoki  wrote:

> On Mon, Dec 16, 2019 at 6:25 PM Inada Naoki 
> wrote:
> >
> > +1 for 1 and 2.
> >
>
> If we find it broke some software, we can step back to regular
> deprecation workflow.
> Python 3.9 is still far from beta yet.  That's why I'm +1 on these
> proposals.
>
> --
> Inada Naoki  
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HWNLBBHSVB5NRQC6ESQQNCQQ2EYUMW27/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YOOL6KM6JQWPSJ5O65IXWERIIDVPD3RU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Serhiy Storchaka wrote:
> It is not more confusing that returning "". By
> default str() returns the same as repr(), unless we made the object
> having other string representation.

Yeah, I suppose not. But that does raise of question of why bytes objects
were made to have a specific form of string representation in the first
place, instead of the generic object address repr. I suspect that it might
be for historical or arbitrary reasons.

But, that's likely an entirely different topic. I'll leave it at that so I
don't derail the main topic.

Serhiy Storchaka wrote:
> You can get an error here if you run Python with -bb. This is a
> temporary option to catch common errors of porting from Python 2.

Huh, interesting.



On Mon, Dec 16, 2019 at 3:59 AM Serhiy Storchaka 
wrote:

> 16.12.19 02:55, Kyle Stanley пише:
> > I'd much prefer to see something like this:
> >  >>> str(b'\xc3\xa1')
> > ...
> > TypeError: bytes argument without an encoding
> >
> > Is there some use case for returning "b'\\xc3\\xa1'" from this operation
> > that I'm not seeing? To me, it seems equally, if not more confusing and
> > pointless than returning an empty string from str(errors='strict') or
> > some other combination of *errors* and *encoding* kwargs without passing
> > an object.
>
> It is not more confusing that returning "". By
> default str() returns the same as repr(), unless we made the object
> having other string representation.
>
> You can get an error here if you run Python with -bb. This is a
> temporary option to catch common errors of porting from Python 2.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RRG4Q7BQWLIYNYNKJGE4BASFWTQ3P7PK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Y4EPTAME4QRLLBGQBSH6YYADBYGPMLMV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Eric V. Smith wrote:
> Debugging. I sometimes do things like: print('\n'.join(str(thing) for
thing in lst)), or various variations on this. This is especially useful >
when maybe something in the list is a bytes object where I was expecting a
string.
>
> I'm not saying it's the best practice, but calling str() on an object is
a currently a guaranteed way of making a string out of it, and I
> don't think we can change it.

I could see that being useful actually. Regardless of "best practices",
it's reasonably common to indiscriminately convert a large sequence of
objects into strings for basic inspection purposes. There may be better
means of debugging, but I wouldn't want to prevent that option entirely for
bytes objects.

But, I suspect that backwards compatibility might be too much of a concern
here for the change to be worthwhile either way. Adding the TypeError or
even gradual deprecation would more than likely lead to a decent amount of
code breakage and maintenance; and changing it to implicitly perform a
UTF-8 encoding would very likely cause some confusion and debugging
difficulties for those who frequently inspect via string conversion.

Thanks for the insight.

On Mon, Dec 16, 2019 at 3:43 AM Eric V. Smith  wrote:

> On 12/16/2019 3:05 AM, Kyle Stanley wrote:
>
> Chris Angelico wrote:
> > ANY object can be passed to str() in order to get some sort of valid
> > printable form. The awkwardness comes from the fact that str()
> > performs double duty - it's both "give me a printable form of this
> > object" and "decode these bytes into text".
>
> While it does make sense for str() to be able to give some form of
> printable form for any object, I suppose that I just don't consider something
> like this:  "b'\\xc3\\xa1'" to be overly useful, at least for any practical
> purposes. Can anyone think of a situation where you would want a string
> representation of a bytes object instead of decoding it?
>
> Debugging. I sometimes do things like: print('\n'.join(str(thing) for
> thing in lst)), or various variations on this. This is especially useful
> when maybe something in the list is a bytes object where I was expecting a
> string.
>
> I'm not saying it's the best practice, but calling str() on an object is a
> currently a guaranteed way of making a string out of it, and I don't think
> we can change it.
>
> Eric
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5B46FTPOWDWNWDD7UL2HCDGSVPCSUUR3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IRF6PY33LQAQXGBGBEHVWZFOAUQV7J6D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Chris Angelico wrote:
> ANY object can be passed to str() in order to get some sort of valid
> printable form. The awkwardness comes from the fact that str()
> performs double duty - it's both "give me a printable form of this
> object" and "decode these bytes into text".

While it does make sense for str() to be able to give some form of
printable form for any object, I suppose that I just don't consider something
like this:  "b'\\xc3\\xa1'" to be overly useful, at least for any practical
purposes. Can anyone think of a situation where you would want a string
representation of a bytes object instead of decoding it?

If not, I think it would be more useful for it to either:

1) Raise a TypeError, assume that the user wanted to decode the string but
forgot to specify an encoding
2) Implicitly decode the bytes object as UTF-8, assume the user meant
str(bytes_obj, encoding='utf-8')

Personally, I'm more in favor of (1) since it's much more explicit and
obvious, but I think (2) would at least be more useful than the current
behavior.

On Sun, Dec 15, 2019 at 8:13 PM Chris Angelico  wrote:

> On Mon, Dec 16, 2019 at 12:00 PM Kyle Stanley  wrote:
> > On a related note though, I'm not a fan of this behavior:
> > >>> str(b'\xc3\xa1')
> > "b'\\xc3\\xa1'"
> >
> > Passing a bytes object to str() without specifying an encoding seems
> like a mistake, I honestly don't see how this ("b'\\xc3\\xa1'") would even
> be useful in any capacity. I would expect this to instead raise a
> TypeError, similar to passing a string to bytes() without specifying an
> encoding:
> > >>> bytes('á')
> > ...
> > TypeError: string argument without an encoding
> >
> > I'd much prefer to see something like this:
> > >>> str(b'\xc3\xa1')
> > ...
> > TypeError: bytes argument without an encoding
> >
> > Is there some use case for returning "b'\\xc3\\xa1'" from this operation
> that I'm not seeing? To me, it seems equally, if not more confusing and
> pointless than returning an empty string from str(errors='strict') or some
> other combination of *errors* and *encoding* kwargs without passing an
> object.
> >
>
> ANY object can be passed to str() in order to get some sort of valid
> printable form. The awkwardness comes from the fact that str()
> performs double duty - it's both "give me a printable form of this
> object" and "decode these bytes into text". With an actual bytes
> object, I always prefer b.decode(...) to str(b, encoding=...). But the
> one-arg form of str() needs to be able to represent a bytes object in
> some way, just as it can represent an int, a Fraction, or a list.
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZP7SXIDQOQVKUF66NVZPS3O4FN3A6DWA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MBP2ZHCGV44IUP4LQEXO2UFEVJX6QNGO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Parameters of str(), bytes() and bytearray()

[Kyle Stanley]

Serhiy Storchaka wrote:
> Forbids calling str() without object if encoding or errors are
> specified. It is very unlikely that this can break a real code, so I
> propose to make it an error without a deprecation period.

+1, I suspect that nobody would intentionally pass an argument to the
encoding and/or errors parameter(s) without specifying an object. Returning
an empty string from this seems like it would cover up bugs rather than be
useful in any capacity.

Serhiy Storchaka wrote:
> 2. Make the first parameter of str(), bytes() and bytearray()
> positional-only.

+1, I don't think I've ever seen a single instance of code that passes the
first parameter, *object*, as a kwarg: str(object=obj). As long as the
other two parameters, *encoding* and *error*, remain keyword arguments, I
think this would make sense.

Serhiy Storchaka wrote:
> 3. Make encoding required if errors is specified in str(). This will
> reduce the number of possible combinations, makes str() more similar to
> bytes() and bytearray() and simplify the mental model: if encoding is
> specified, then we decode, and the first argument must be a bytes-like
> object, otherwise we convert an object to a string using __str__.

Hmm, I think this one might require some further consideration. But I will
say that the implicit behavior is not very obvious.

Isn't overly clear, implicit 'utf-8' conversion:
>>> str(b'\xc3\xa1', errors='strict')
'á'

Makes sense, and is highly explicit:
>>> str(b'\xc3\xa1', encoding='utf-8', errors='strict')
'á'

This is also fine ('strict' is a very reasonable default for *errors*)
>>> str(b'\xc3\xa1', encoding='utf-8')
'á'


On a related note though, I'm not a fan of this behavior:
>>> str(b'\xc3\xa1')
"b'\\xc3\\xa1'"

Passing a bytes object to str() without specifying an encoding seems like a
mistake, I honestly don't see how this ("b'\\xc3\\xa1'") would even be
useful in any capacity. I would expect this to instead raise a TypeError,
similar to passing a string to bytes() without specifying an encoding:
>>> bytes('á')
...
TypeError: string argument without an encoding

I'd much prefer to see something like this:
>>> str(b'\xc3\xa1')
...
TypeError: bytes argument without an encoding

Is there some use case for returning "b'\\xc3\\xa1'" from this operation
that I'm not seeing? To me, it seems equally, if not more confusing and
pointless than returning an empty string from str(errors='strict') or some
other combination of *errors* and *encoding* kwargs without passing an
object.

On Sun, Dec 15, 2019 at 9:10 AM Serhiy Storchaka 
wrote:

> Currently str() takes up to 3 arguments. All are optional and
> positional-or-keyword. All combinations are valid:
>
> str()
> str(object=object)
> str(object=buffer, encoding=encoding)
> str(object=buffer, errors=errors)
> str(object=buffer, encoding=encoding, errors=errors)
> str(encoding=encoding)
> str(errors=errors)
> str(encoding=encoding, errors=errors)
>
> The last three are especially surprising. If you do not specify an
> object, str() ignores values of encoding and errors and returns an empty
> string.
>
> bytes() and bytearray() are more limited. Valid combinations are:
>
> bytes()
> bytes(source=object)
> bytes(source=string, encoding=encoding)
> bytes(source=string, encoding=encoding, errors=errors)
>
> I propose several changes:
>
> 1. Forbids calling str() without object if encoding or errors are
> specified. It is very unlikely that this can break a real code, so I
> propose to make it an error without a deprecation period.
>
> 2. Make the first parameter of str(), bytes() and bytearray()
> positional-only. Originally this feature was an implementation artifact:
> before 3.6 parameters of a C implemented function should be either all
> positional-only (if used PyArg_ParseTuple), or all keyword (if used
> PyArg_ParseTupleAndKeywords). So str(), bytes() and bytearray() accepted
> the first parameter by keyword. We already made similar changes for
> int(), float(), etc: int(x=42) no longer works.
>
> Unlikely str(object=object) is used in a real code, so we can skip a
> deprecation period for this change too.
>
> 3. Make encoding required if errors is specified in str(). This will
> reduce the number of possible combinations, makes str() more similar to
> bytes() and bytearray() and simplify the mental model: if encoding is
> specified, then we decode, and the first argument must be a bytes-like
> object, otherwise we convert an object to a string using __str__.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YMIGWRUERUG66CKRJXDXNPCIDHRQJY6V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To 

[Python-Dev] Re: Travis CI for backports not working.

[Kyle Stanley]

> I.e. if someone breaks some test, is Travis-CI the only thing that keeps
the breakage from landing on master?

We still have Azure Pipelines as a mandatory check for PRs before they can
be merged to master, which includes a few additional platforms and hasn't
had any recent issues (AFAIK). I believe Steve Dower was also recently
experimenting with adding GitHub Actions as an additional CI service.

On Thu, Dec 12, 2019 at 10:16 PM Guido van Rossum  wrote:

> Does Travis-CI serve any purpose for us still? I.e. if someone breaks some
> test, is Travis-CI the only thing that keeps the breakage from landing on
> master?
>
> On Thu, Dec 12, 2019 at 5:46 PM Kyle Stanley  wrote:
>
>> Victor Stinner wrote:
>> > What is the issue? Can someone please open a bug report at
>> https://bugs.python.org/ so I can try to investigate?
>>
>> From my understanding, it looks to be pyenv related and not something we
>> can fix on our end, at least based on the build logs:
>> https://travis-ci.org/python/cpython/jobs/624160244?utm_medium=notification_source=github_status.
>> This was from a recent backport PR to 3.7 (backport also failing for 3.8
>> with similar issues).
>>
>> On Thu, Dec 12, 2019 at 8:14 PM Victor Stinner 
>> wrote:
>>
>>> What is the issue? Can someone please open a bug report at
>>> https://bugs.python.org/ so I can try to investigate?
>>>
>>> Victor
>>>
>>> Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
>>> >
>>> > This is failing again, so I had to switch off Travis from being a
>>> requirement (again).
>>> >
>>> > I'm not not going to flip it back on until Travis has been stable for
>>> a month as I don't like being the blocker on stuff when I can help it. And
>>> if Travis isn't stable in a month then we might need to start talking about
>>> turning it off entirely as flaky CI is never useful.
>>> > ___
>>> > Python-Dev mailing list -- python-dev@python.org
>>> > To unsubscribe send an email to python-dev-le...@python.org
>>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> > Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
>>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>>
>>> --
>>> Night gathers, and now my watch begins. It shall not end until my death.
>>> ___
>>> Python-Dev mailing list -- python-dev@python.org
>>> To unsubscribe send an email to python-dev-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-dev@python.org/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/34HWMTFBGXFGAAN5LCHHGX3PMO4CGZBT/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TSM5LJZS2AJSFVIEW64PRJXEK7T5J3EM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

[Kyle Stanley]

Victor Stinner wrote:
> What is the issue? Can someone please open a bug report at
https://bugs.python.org/ so I can try to investigate?

>From my understanding, it looks to be pyenv related and not something we
can fix on our end, at least based on the build logs:
https://travis-ci.org/python/cpython/jobs/624160244?utm_medium=notification_source=github_status.
This was from a recent backport PR to 3.7 (backport also failing for 3.8
with similar issues).

On Thu, Dec 12, 2019 at 8:14 PM Victor Stinner  wrote:

> What is the issue? Can someone please open a bug report at
> https://bugs.python.org/ so I can try to investigate?
>
> Victor
>
> Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
> >
> > This is failing again, so I had to switch off Travis from being a
> requirement (again).
> >
> > I'm not not going to flip it back on until Travis has been stable for a
> month as I don't like being the blocker on stuff when I can help it. And if
> Travis isn't stable in a month then we might need to start talking about
> turning it off entirely as flaky CI is never useful.
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/34HWMTFBGXFGAAN5LCHHGX3PMO4CGZBT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

Antoine Pitrou wrote:
> 1M concurrent database transactions?  Does that sound reasonable at
> all?  Your database administrator probably won't like you.

I agree that 1M concurrent transactions would not be reasonable for the
vast majority of database configurations, I didn't mean to specifically
imply that 1M would be something reasonable today. That's why I said "a
large number of concurrent transactions" instead of specifically saying "1M
concurrent transactions". I honestly don't know if any databases are close
to capable of handling that many transactions at once, at least not at the
present time.

But, although we may think it's ridiculous today, who's to say that won't
be an occurrence in the future? Imagine a scenario where there was a
massive super-cluster of database servers, that performed real-time update
transactions every time a single item was checked in and out of some global
inventory system.

Currently, something like this would likely have to be implemented through
batching, where x number of updates have to be queued or x amount of time
has to pass before the next transaction is started. Alternatively, each
facility or region would have its own local database that synchronizes with
the global database every so often. But, there could be a significant
advantage in having a near-perfectly synchronized global inventory system,
which would only be possible if it was updated in real-time. IMO, the max
number of concurrent transactions that the a database system can handle at
once is a very clear application of Moore's Law.

My point being is that I don't want to arbitrarily restrict how many
coroutine objects can exist at once without having a strong reason for
doing so AND having a limit that's reasonable in the long term. 1M would
have the advantage of being easy to remember, but other than that I see no
reason why that should specifically be the limit for the max number of
coroutines. As Guido mentioned at the start of the thread, a coroutine
object is "just another Python object and has no operating resources
associated with it". Other than main memory usage, there's no other
external limit to the max number of coroutine objects that can exist at
once.

Note: Although coroutines were already dropped from PEP 611, I felt that
this response was still worthwhile to write. I suspect that the topic of
"coroutine object limits" is likely to come up again in the future.


On Wed, Dec 11, 2019 at 4:46 PM Antoine Pitrou  wrote:

> On Mon, 9 Dec 2019 21:42:36 -0500
> Kyle Stanley  wrote:
>
> > > (b) Why limit coroutines? It's just another Python object and has no
> > operating resources associated with it. Perhaps your definition of
> > coroutine is different, and you are thinking of OS threads?
> >
> > This was my primary concern with the proposed PEP. At the moment, it's
> > rather trivial to create one million coroutines, and the total memory
> taken
> > up by each individual coroutine object is very minimal compared to each
> OS
> > thread.
> >
> > There's also a practical use case for having a large number of coroutine
> > objects, such as for asynchronously:
> >
> > 1) Handling a large number of concurrent clients on a continuously
> running
> > web server that receives a significant amount of traffic.
>
> Not sure how that works?  Each client has an accepted socket, which is
> bound to a local port number, and there are 65536 TCP port numbers
> available.  Unless you're using 15+ coroutines per client, you probably
> won't reach 1M coroutines that way.
>
> > 2) Sending a large number of concurrent database transactions to run on a
> > cluster of database servers.
>
> 1M concurrent database transactions?  Does that sound reasonable at
> all?  Your database administrator probably won't like you.
>
> > something like this definitely scales over time. Arbitrarily placing a
> > limit on the total number of coroutine objects doesn't make sense to me
> for
> > that reason.
>
> There are a lot of arbitrary limits inside a computer system.  You just
> aren't aware of them because you don't hit them in practice.  Claiming
> that limits shouldn't exist is just pointless.
>
> Regards
>
> Antoine.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/O3ZODXHEIJ2SM5SZBOVJ4PIAQMSYNXEJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PYUSGK4UDMGGDJPQ6E6YHKG2O5D5DTEL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

Andrew Svetlov wrote:
> > Not sure how that works?  Each client has an accepted socket, which is
> > bound to a local port number, and there are 65536 TCP port numbers
> > available.  Unless you're using 15+ coroutines per client, you probably
> > won't reach 1M coroutines that way.
>
> I'm sorry, but the accepted socket has the same local port number as
> the listening one.
> Routing is performed by (local_ip, local_port, remote_ip, remote_port)
quad.

IIRC, that combination is typically referred to as the "port address" (IP
addr + port num source, IP addr + port num destination). All four are are
required in TCP, and in UDP, the IP and source port are optional. So in
UDP, this could potentially just be (remote_ip, remote_port).

Also, it's possible to bind multiple AF_INET (or AF_INET6) sockets to a
single port address by using the SO_REUSEPORT socket option, which we
discussed recently in bpo-37228 (https://bugs.python.org/issue37228). The
only requirement is that the same UID is used for each socket bound to the
same port address (from my understanding, SO_REUSEPORT is typically used
for binding a single process to multiple listening sockets).

TL;DR: It's definitely possible to have more than one client per TCP port.

I'll admit that I've never personally seen production code that uses
anywhere near 1M coroutine objects, but we shouldn't limit users from doing
that without a good reason. At the present moment, it's rather trivial to
create 1M coroutine objects on any system with ~1.3GB+ available main
memory (see my code example in
https://mail.python.org/archives/list/python-dev@python.org/message/WYZHKRGNOT252O3BUTFNDVCIYI6WSBXZ/
).

There's also the infamous "10M" problem, of accepting 10 million concurrent
clients without significant performance issues. This is mostly theoretical
at the moment, but there's an article that explains how the problem could
be addressed by using 10M goroutines: https://goroutines.com/10m. I see no
reason why this couldn't be potentially translated into Python's
coroutines, with the usage of an optimized event loop policy such as uvloop.

But, either way, Mark Shannon removed the 1M coroutine limit from PEP 611,
due to it having the least strong argument out of all of the proposed
limits and a significant amount of push-back from the dev community.

Andrew Svetlov wrote:
> The listening socket can accept hundreds of thousands of concurrent
> client connections.
> The only thing that should be tuned for this is increasing the limit
> of file descriptors.

The default soft limit of file descriptors per process on Linux is 1024
(which can be increased), but you could exceed a per-process limitation of
file descriptors by forking child processes. I'm have no idea what the
realistic maximum limit of global FDs would be for most modern servers
though, but here's the upper bound limit on Linux kernel 5.3.13:

[aeros:~]$ cat /proc/sys/fs/file-max
9223372036854775807

My system's current hard limit of file descriptors is much lower, but is
still fairly substantial:

[aeros:~]$ ulimit -nH
524288

I recall reading somewhere that per additional 100 file descriptors, it
requires approximately 1MB of main memory. Based on that estimate, 1M FDs
would require ~10GB+. This doesn't seem unreasonable to me, especially on a
modern server. But I'd imagine the actual memory usage depends upon how
much data is being buffered at once through the pipes associated with each
FD, but I believe this can be limited through the FD option F_SETPIPE_SZ (
https://linux.die.net/man/2/fcntl).

Note: I was unable to find a credible source on the minimum memory usage
per additional FD, so clarification on that would be appreciated.

On Wed, Dec 11, 2019 at 5:06 PM Andrew Svetlov 
wrote:

> On Wed, Dec 11, 2019 at 11:52 PM Antoine Pitrou 
> wrote:
> >
> > On Mon, 9 Dec 2019 21:42:36 -0500
> > Kyle Stanley  wrote:
> >
> > >
> > > There's also a practical use case for having a large number of
> coroutine
> > > objects, such as for asynchronously:
> > >
> > > 1) Handling a large number of concurrent clients on a continuously
> running
> > > web server that receives a significant amount of traffic.
> >
> > Not sure how that works?  Each client has an accepted socket, which is
> > bound to a local port number, and there are 65536 TCP port numbers
> > available.  Unless you're using 15+ coroutines per client, you probably
> > won't reach 1M coroutines that way.
> >
>
> I'm sorry, but the accepted socket has the same local port number as
> the listening one.
> Routing is performed by (local_ip, local_port, remote_ip, remote_port)
> quad.
>
> The listening socket can accept hundreds of thousands of concurrent
> client connections.
> The only thing that should be tuned for this is inc

[Python-Dev] Re: Dropping coroutines from PEP 611.

[Kyle Stanley]

With the removal of the coroutine limit, I think that PEP 611 will be a lot
easier to consider. Personally, I've gone from -1 to +0. But, with the
class limit also being rather controversial, I think the PEP would benefit
significantly from backing that up with some form of analysis on memory
usage in existing code.

For example, in addition to the existing section "Total number of classes
in a running interpreter", where it mentions a potential reduction in
overhead on a Python object from 40 bytes to 16 bytes: I think this
argument would be much stronger with an example of production code. You
could perform a rough estimate on how much memory is currently being used
on objects in the code example, and mention how much could be potentially
saved by imposing the 1M class limit.

In general though, I think adding some practical examples to the PEP and
explaining how they could benefit from the limits would make it much more
convincing. At the present moment, it might be a bit too abstract.

On Wed, Dec 11, 2019 at 5:19 AM Mark Shannon  wrote:

> Hi all,
>
> Thanks again for all your feedback on PEP 611.
> So far the biggest pushback on limits has been for the number of
> classes, line count and coroutines.
>
> Of those, I have been convinced that a limit on the number of coroutines
> would the hardest to workaround. In addition the justification for
> limiting coroutines is probably the weakest.
>
> Consequently, I'm dropping the limit on the number of coroutines from
> the PEP.
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/6ESJKTW4CP42KWJVSEEYJABGU73T4UPA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AK7XRUXJMFKGR6MKYHQM4HQGI57BZFMA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

Steve D'Aprano wrote:
> As I understand the PEP, the proposal is to change a bunch of C-level
> structs which currently contain 32-bit fields into 20-bit fields. To
> change that, you would need to recompile with new struct definitions and
> build a new interpreter binary.
>
> If I'm right, making this configurable at *build-time* could be
> possible, but that's probably a bad idea. That's like the old "narrow
> versus wide Unicode" builds. It doubles (at least!) the amount of effort
> needed to maintain Python, for something which (if the PEP is correct)
> benefits nearly nobody but costs nearly everyone.

Ah, if that's case I'll withdraw the suggestion for allowing the limits to
be configurable through sys then. Thanks for the clarification.

I have some basic understanding of how C structs work, but I'll admit that
it's far from an area that I'm knowledgeable about. My feedback is mostly
based on experience as a user and involvement with stdlib development, not
as a C developer. I don't have a strong understanding of the implementation
details behind the limits.

Although if they're not (reasonably) configurable by most users, I would
consider that to further reinforce my previous statement that we should
have some form of concrete evidence; to prove that imposing the limits will
provide tangible benefits to the vast majority of Python users.

On Tue, Dec 10, 2019 at 4:45 AM Steven D'Aprano  wrote:

> On Mon, Dec 09, 2019 at 11:27:56PM -0500, Kyle Stanley wrote:
>
> > I agree, I think that sys would likely be the most reasonable place to
> read
> > these limits from. Also, it seems like a good location for setting of the
> > limits, if that becomes an option. This would go along well with the
> > existing sys.getrecursionlimit() and sys.setrecursionlimit().
> >
> > In general, this proposal would be much easier to consider if the limits
> > were customizable. I'm not sure if it would be reasonable for all of the
> > options, but it would at least allow those who have a legitimate use case
> > for going beyond the limits (either now or in the future) to still be
> able
> > to do so.
>
> Someone will correct me if I'm wrong, but my reading of the PEP tells me
> that these are structural limits in the interpreter's internal data
> structures, so they can't be configured at runtime.
>
> The impression I get is that you believe that the proposal is to add a
> bunch of runtime checks like this:
>
> # pseudo-code for the module-loading code
> lines = 0;
> while 1
> {
> read and process line of code
> lines += 1;
> if lines >= 100:
> GOTO error condition
> }
>
> In that case, it would be easy to change the 100 constant to a value
> that can be configured at runtime. You wouldn't even need to quit the
> running interpreter.
>
> As I understand the PEP, the proposal is to change a bunch of C-level
> structs which currently contain 32-bit fields into 20-bit fields. To
> change that, you would need to recompile with new struct definitions and
> build a new interpreter binary.
>
> If I'm right, making this configurable at *build-time* could be
> possible, but that's probably a bad idea. That's like the old "narrow
> versus wide Unicode" builds. It doubles (at least!) the amount of effort
> needed to maintain Python, for something which (if the PEP is correct)
> benefits nearly nobody but costs nearly everyone.
>
>
>
> --
> Steven
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/Q2UOEO3VOLG762WJ5N7LMJGX6ULUE6JR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V5T3GYC5UB4HICCSNHXXUKIZ4KV4YWQC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

[Kyle Stanley]

Chris Angelico wrote:
> We have people who believe that a bit
> mask will slow things down, others who claim that improved cache
> locality will speed things up, and Mark asks us to please justify our
> objections with numbers. But surely it's up to Mark to show numbers
> first?

+1. While it would be helpful for criticisms of the PEP to be more specific
and provide more of a cost-benefit analysis, the burden of proof for
demonstrating the benefits provided are ultimately up to the author(s) of
the PEP. We require far more justification for making a change to impose
the limits in the first place than we do for maintaining the status quo.

Personally, I don't think it would be reasonable to impose any of these
limits without some form of concrete evidence that doing so will provide an
improvement; in any combination of efficiency, performance, and/or
security, as the PEP suggests would happen. But, I don't think exact
numbers are needed. (I.E. adding these limits will improve performance
across all Python programs by an average of 22.359%!).

On Mon, Dec 9, 2019 at 7:15 PM Chris Angelico  wrote:

> On Tue, Dec 10, 2019 at 10:51 AM Steven D'Aprano 
> wrote:
> >
> > On Mon, Dec 09, 2019 at 02:12:37PM -0800, Nathaniel Smith wrote:
> > > > > On 09/12/2019 2:15 pm, Chris Angelico wrote:
> > > > You: "We should limit things. Stuff will be faster."
> > > > Others: "Really? Because bit masking is work. It'll be slower."
> >
> > I'm not an expert, but the impression I've got from various discussions
> > on performance over the last few years is that the single biggest
> > bottleneck for CPU performance is memory locality. Cache misses are so
> > expensive, and CPU instructions so fast, that memory locality is king
> > and the cost of bit masking is insignificant. In other words, worrying
> > about the cost of bit masking in C code is so very 1990s.
> >
> > I could be wrong of course: I'm not an expert. And I don't think we
> > should take it for granted that this is the case, unless some experts on
> > modern CPUs speak up and say that Mark is so obviously correct that a
> > demonstration is unnecessary.
>
> And the speculation continues.
>
> > > > You: "Maybe we limit it somewhere else, whatever. It'll be faster."
> >
> > That's a totally unfair and inaccurate representation of Mark's
> > position. The PEP doesn't say "let's put in arbitrary limits in random
> > places for the lols", he proposed seven concrete limits and gave reasons
> > for why he expects that they will improve memory efficiency, safety,
> > performance or all three.
>
> *He expects that*. That's what I'm talking about. We have lots of
> speculation and no evidence either way.
>
> > Having said that though, I think you are right that the PEP could do
> > with a bit more detail on the current status quo and existing limits,
> > and how the proposed changes will improve safety and memory use.
>
> Exactly. Yes, I know that I massively oversimplified things in that
> post. But you nonetheless acknowledge here that we are *still* quite
> lacking in any actual evidence. We have people who believe that a bit
> mask will slow things down, others who claim that improved cache
> locality will speed things up, and Mark asks us to please justify our
> objections with numbers. But surely it's up to Mark to show numbers
> first?
>
> ChrisA
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YN2BDJFPGHTEZJRDN7LXNDAGXWXB6XRA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EDZOEKHLF52MG3JJ6HIVHLCGXQJRIM34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

> I also would suggest for PEP 611 that any limits are discoverable (maybe
in sys) so it can be used by other implementations like Jython.

I agree, I think that sys would likely be the most reasonable place to read
these limits from. Also, it seems like a good location for setting of the
limits, if that becomes an option. This would go along well with the
existing sys.getrecursionlimit() and sys.setrecursionlimit().

In general, this proposal would be much easier to consider if the limits
were customizable. I'm not sure if it would be reasonable for all of the
options, but it would at least allow those who have a legitimate use case
for going beyond the limits (either now or in the future) to still be able
to do so.

On Mon, Dec 9, 2019 at 8:51 PM Jim Baker  wrote:

> I was thinking the same thing. We should distinguish limits with respect
> to the codegen process, which seem reasonable, vs runtime. Classes and
> coroutines are objects, and like objects in general, the program should
> have the option of filling its heap with any arbitrary objects. (Whether
> wise or not, this design is not for us to arbitrarily limit. For example, I
> recall that Eve Online is/was running large numbers of stackless
> coroutines, possibly well in excess of 1M.)
>
> For some comparison:
> Note the JVM has it made easier to tune the use of the native heap for
> class objects since Java 8, in part to relax earlier constraints around
> "permgen" allocation - by default, class objects are automatically
> allocated from the heap without limit (this is managed by "metaspace"). I
> suppose if this was a tunable option, maybe it could be useful, but
> probably not - Java's ClassLoader design is prone to leaking classes, as we
> know from our work on Jython. There's nothing comparable to my knowledge
> for why this would be the case for CPython class objects more than other
> objects.
>
> I also would suggest for PEP 611 that any limits are discoverable (maybe
> in sys) so it can be used by other implementations like Jython. There's no
> direct correspondence between LOC and generated Python or Java bytecode,
> but it could possibly still be helpful for some codegen systems. Jython is
> limited to 2**15 bytes per method due to label offsets, although we do have
> workarounds for certain scenarios, and could always compile, then run
> Python bytecode for large methods. (Currently we use CPython to do that
> workaround compilation, thanks!)
>
> Lastly, PEP 611 currently erroneously conjectures that "For example,
> Jython might need to use a lower class limit of fifty or sixty thousand
> becuase of JVM limits."
>
> - Jim
>
>
> On Mon, Dec 9, 2019 at 9:55 AM Guido van Rossum  wrote:
>
>> I want to question two specific limits.
>>
>> (a) Limiting the number of classes, in order to potentially save space in
>> object headers, sounds like a big C API change, and I think it's better to
>> lift this idea out of PEP 611 and debate the and cons separately.
>>
>> (b) Why limit coroutines? It's just another Python object and has no
>> operating resources associated with it. Perhaps your definition of
>> coroutine is different, and you are thinking of OS threads?
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> 
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/CJO36YRFWCTEUUROJVXIQDMWGZBFAD5T/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RQIVRB4YHNQZ2M7GCXRRCOZHZSOSNSZL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S22M4XH6PY6VKH446FDWC4JT2PFEDHL2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

> This logic doesn't seem much different than would be for coroutines...
Just need to wait for larger systems...

> With 100k threads started we're only using 8G memory, there are plenty of
systems today with more than 80G of RAM

Well, either way, I think it's still a solid argument against imposing the
1M limit on coroutines. Arguing in favor or against 1M OS threads wasn't
the primary message I was trying to convey, it was just to demonstrate that
1M coroutines could be created and awaited concurrently on most current
systems (w/ ~1.3GB+ available RAM and virtual memory).

But I think there's a reasonable question of practicality when it comes to
running 1M OS threads simultaneously. For especially high volumes of
concurrent tasks, OS threads are generally not the best solution (for
CPython, at least). They work for handling a decent number IO-bound tasks
such as sending out and processing network requests, but coroutine objects
are significantly more efficient when it comes to memory usage.

For the usage of child processes, we have watcher implementations that
don't use OS threads at all, such as the recently added PidfdChildWatcher (
https://docs.python.org/3.9/library/asyncio-policy.html#asyncio.PidfdChildWatcher).
There are also others that don't spawn a new thread per process.

That being said, you are correct in that at some point, the memory usage
for running 1M simultaneous OS threads will be perfectly reasonable. I'm
just not sure if there's a practical reason to do so, considering that more
efficient means of implementing parallelism are available when memory usage
becomes a significant concern.

Of course, the main question here is: "What benefit would imposing this
particular limit to either coroutines objects or OS threads provide?".

Personally, I'm not entirely convinced that placing a hard limit of 1M at
once for either would result in a significant benefit to performance,
efficiency, or security (mentioned in the PEP, as a reason for imposing the
limits). I could see it being more useful for other areas though, such as
lines of code or bytecode instructions per object.

I just think that placing a limit of 1M on current coroutine objects would
not be reasonable. But between the two, I think a limit of 1M on OS threads
is *more* reasonable in comparison.


On Mon, Dec 9, 2019 at 10:17 PM Khazhismel Kumykov  wrote:

>
>
> On Mon, Dec 9, 2019, 18:48 Kyle Stanley  wrote:
>
>> > (b) Why limit coroutines? It's just another Python object and has no
>> operating resources associated with it. Perhaps your definition of
>> coroutine is different, and you are thinking of OS threads?
>>
>> This was my primary concern with the proposed PEP. At the moment, it's
>> rather trivial to create one million coroutines, and the total memory taken
>> up by each individual coroutine object is very minimal compared to each OS
>> thread.
>>
>> There's also a practical use case for having a large number of coroutine
>> objects, such as for asynchronously:
>>
>> 1) Handling a large number of concurrent clients on a continuously
>> running web server that receives a significant amount of traffic.
>> 2) Sending a large number of concurrent database transactions to run on a
>> cluster of database servers.
>>
>> I don't know that anyone is currently using production code that results
>> in 1 million coroutine objects within the same interpreter at once, but
>> something like this definitely scales over time. Arbitrarily placing a
>> limit on the total number of coroutine objects doesn't make sense to me for
>> that reason.
>>
>> OS threads on the other hand take significantly more memory. From a
>> recent (but entirely unrelated) discussion where the memory usage of
>> threads was brought up, Victor Stinner wrote a program that demonstrated
>> that each OS thread takes up approximately ~13.2kB on Linux, which I
>> verified on kernel version 5.3.8. See https://bugs.python.org/msg356596.
>>
>> For comparison, I just wrote a similar program to compare the memory
>> usage between 1M threads and 1M coroutines:
>>
>> ```
>> import asyncio
>> import threading
>> import sys
>> import os
>>
>> def wait(event):
>> event.wait()
>>
>> class Thread(threading.Thread):
>> def __init__(self):
>> super().__init__()
>> self.stop_event = threading.Event()
>> self.started_event = threading.Event()
>>
>> def run(self):
>> self.started_event.set()
>> self.stop_event.wait()
>>
>> def stop(self):
>> self.stop_event.set()
>> self.join()
>>
>> def display_rss():
>> os.system(f"grep ^VmRSS /pro

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

[Kyle Stanley]

> (b) Why limit coroutines? It's just another Python object and has no
operating resources associated with it. Perhaps your definition of
coroutine is different, and you are thinking of OS threads?

This was my primary concern with the proposed PEP. At the moment, it's
rather trivial to create one million coroutines, and the total memory taken
up by each individual coroutine object is very minimal compared to each OS
thread.

There's also a practical use case for having a large number of coroutine
objects, such as for asynchronously:

1) Handling a large number of concurrent clients on a continuously running
web server that receives a significant amount of traffic.
2) Sending a large number of concurrent database transactions to run on a
cluster of database servers.

I don't know that anyone is currently using production code that results in
1 million coroutine objects within the same interpreter at once, but
something like this definitely scales over time. Arbitrarily placing a
limit on the total number of coroutine objects doesn't make sense to me for
that reason.

OS threads on the other hand take significantly more memory. From a recent
(but entirely unrelated) discussion where the memory usage of threads was
brought up, Victor Stinner wrote a program that demonstrated that each OS
thread takes up approximately ~13.2kB on Linux, which I verified on kernel
version 5.3.8. See https://bugs.python.org/msg356596.

For comparison, I just wrote a similar program to compare the memory usage
between 1M threads and 1M coroutines:

```
import asyncio
import threading
import sys
import os

def wait(event):
event.wait()

class Thread(threading.Thread):
def __init__(self):
super().__init__()
self.stop_event = threading.Event()
self.started_event = threading.Event()

def run(self):
self.started_event.set()
self.stop_event.wait()

def stop(self):
self.stop_event.set()
self.join()

def display_rss():
os.system(f"grep ^VmRSS /proc/{os.getpid()}/status")

async def test_mem_coros(count):
print("Coroutine memory usage before:")
display_rss()
coros = tuple(asyncio.sleep(0) for _ in range(count))
print("Coroutine memory usage after creation:")
display_rss()
await asyncio.gather(*coros)
print("Coroutine memory usage after awaiting:")
display_rss()

def test_mem_threads(count):
print("Thread memory usage before:")
display_rss()
threads = tuple(Thread() for _ in range(count))
print("Thread memory usage after creation:")
display_rss()
for thread in threads:
thread.start()
print("Thread memory usage after starting:")
for thread in threads:
thread.run()
print("Thread memory usage after running:")
display_rss()
for thread in threads:
thread.stop()
print("Thread memory usage after stopping:")
display_rss()

if __name__ == '__main__':
count = 1_000_000
arg = sys.argv[1]
if arg == 'threads':
test_mem_threads(count)
if arg == 'coros':
asyncio.run(test_mem_coros(count))

```
Here are the results:

1M coroutine objects:

Coroutine memory usage before:
VmRSS: 14800 kB
Coroutine memory usage after creation:
VmRSS:651916 kB
Coroutine memory usage after awaiting:
VmRSS:   1289528 kB

1M OS threads:

Thread memory usage before:
VmRSS: 14816 kB
Thread memory usage after creation:
VmRSS:   4604356 kB
Traceback (most recent call last):
  File "temp.py", line 60, in 
test_mem_threads(count)
  File "temp.py", line 44, in test_mem_threads
thread.start()
  File "/usr/lib/python3.8/threading.py", line 852, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread

(Python version: 3.8)
(Linux kernel version: 5.13)

As is present in the results above, 1M OS threads can't even be ran at
once, and the memory taken up just to create the 1M threads is ~3.6x more
than it costs to concurrently await the 1M coroutine objects. Based on
that, I think it would be reasonable to place a limit of 1M on the total
number of OS threads. It seems unlikely that a system would be able to
properly handle 1M threads at once anyways, whereas that seems entirely
feasible with 1M coroutine objects. Especially on a high traffic server.

On Mon, Dec 9, 2019 at 12:01 PM Guido van Rossum  wrote:

> I want to question two specific limits.
>
> (a) Limiting the number of classes, in order to potentially save space in
> object headers, sounds like a big C API change, and I think it's better to
> lift this idea out of PEP 611 and debate the and cons separately.
>
> (b) Why limit coroutines? It's just another Python object and has no
> operating resources associated with it. Perhaps your definition of
> coroutine is different, and you are thinking of OS threads?
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 

[Python-Dev] Re: Travis CI for backports not working.

[Kyle Stanley]

Steve Dower wrote:
> GitHub Actions *is* a CI service now, so my PR is actually using their
> machines for Windows/macOS/Ubuntu build and test.

Oh I see, that makes more sense. Thanks for the clarification.

Steve Dower wrote:
> It doesn't change the ease of enabling/disabling anything - that's still
> very easy if you're an administrator on our repo and impossible if
> you're not :)

Yeah I was aware that it required administrative permissions, I was just
under the impression that it wasn't a simple on/off toggle, and that there
wasn't an easy way to ensure failures were occurring due to external
factors. I suppose that's a separate issue though.

Steve Dower wrote:
> And you don't need a separate login to rerun checks - it's just a simple
button in the
> GitHub UI.

That would be especially great, and much better than closing+reopening the
PR. Restarting the checks is typically a last resort since intermittent
regression test failures are a concern, but it's highly useful when the
failure is occurring due to external factors (such as a recently merged PR
or issues with the CI service itself).

On Fri, Dec 6, 2019 at 1:33 PM Steve Dower  wrote:

> On 06Dec2019 1023, Kyle Stanley wrote:
> > Steve Dower wrote:
> >  > As a related aside, I've been getting GitHub Actions support together
> >  > (which I started at the sprints).
> >
> > Would adding support for GitHub Actions make it easier/faster to
> > temporarily disable and re-enable specific CI services when they're
> > having external issues? IIUC, that seems to be the primary concern to
> > address.
> >
> > Note that I'm not particularly well acquainted with GitHub Actions,
> > other than briefly looking over https://github.com/features/actions.
>
> GitHub Actions *is* a CI service now, so my PR is actually using their
> machines for Windows/macOS/Ubuntu build and test.
>
> It doesn't change the ease of enabling/disabling anything - that's still
> very easy if you're an administrator on our repo and impossible if
> you're not :)
>
> However, it does save jumping to an external website to view logs, and
> over time we can improve the integration so that error messages are
> captured properly (but you can easily view each step). And you don't
> need a separate login to rerun checks - it's just a simple button in the
> GitHub UI.
>
> Cheers,
> Steve
>
>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KB7F5JNUILUH3H2DJ3T7NGZAKQXMK76D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Should we require all deprecations to have a removal version that we follow through on?

[Kyle Stanley]

Victor Stinner wrote:
> Isn't it already the current unwritten deprecation process?

Personally, I don't think we should rely on an unwritten process for
something as important and potentially breaking as a deprecation process.
Regardless of the outcome of this discussion, I think we should try to
fully write out the process in the devguide; while still providing some
amount of flexibility.

Victor Stinner wrote:
> I'm not sure about enforcing the removal. Some APIs were deprecated,
> but it was unsure if we really wanted to remove them. Or because we
> had no idea how long it would take for developers to upgrade to the
> new functions.
...
> I would suggest to schedule removal, but not require it too strongly ;-)

Would it be reasonable to require an minimum amount of versions to be
specified (such as n versions ahead), but provide flexibility in terms to
delaying the removal, as needed? IMO, it would be more convenient for users
to have a "minimum removal" version in mind, rather than an unscheduled
deprecation that could have a version+2 (deprecation and removal in 2
versions) assigned at any point in time.

This can be difficult sometimes when we're not sure what n versions from
now will actually be called (such as with 3.10 and 4.0), but I don't think
it would be an issue to state something like this:
"func() has been deprecated, and will be removed in no sooner than 4
versions after 3.9"

instead of:
"func() has been deprecated, and will be removed in 3.13"

for long term deprecations. By "long term", I mean more than two versions.

Victor Stinner wrote:
> I know that it's an unpopular opinion, but I would strongly prefer
> Python 4 to behave exactly as any another Python 3.x release in term
> of deprecation. If I exaggerate, I would prefer that Python 4.0 would
> have *less* incompatible changes than the last Python 3.x release.

My understanding was that we were specifically waiting on considering a
Python 4.0 until there were major enough improvements/changes (such as the
"GILectomy", for example) to justify some degree of incompatibility. If it
would behave exactly the same as a standard 3.x release in terms to
incompatible changes (or have less), what would be the purpose of making a
major version change in the first place? I don't see an issue with
indefinitely continuing the 3.x line in the meantime.

Victor Stinner wrote:
> I like to remove deprecated features at the beginning of a dev cycle
> to get users feedback earlier. If they are removed late in the dev
> cycle, there is a higher risk that a problematic incompatible change
> goes through a 3.x.0 final release and so cannot be reverted anymore.

I agree, the earlier a deprecation occurs in a given release cycle, the
easier it is for users to provide feedback and then potentially prepare for
the change. Perhaps we could establish some form of guideline, for example:
"When possible, the removal of previously deprecated features should occur
as early as possible in a version's dev cycle, preferably during the alpha
phases. This provides users more time to provide feedback and plan for the
potential removal".

On Fri, Dec 6, 2019 at 10:58 AM Victor Stinner  wrote:

> Hi,
>
> Le mer. 27 nov. 2019 à 19:40, Brett Cannon  a écrit :
> > What do people think of the idea of requiring all deprecations
> specifying a version that the feature will be removed in (which under our
> annual release cadence would be at least the third release from the start
> of the deprecation, hence the deprecation being public for 2 years)? And
> that we also follow through with that removal?
>
> Isn't it already the current unwritten deprecation process? The common
> case is to schedule the removal. Previously, it was common to wait 2
> releases before removing anything: pending deprecatation in 3.6,
> deprecation in 3.7, removal in 3.8. It means that developers are
> warned during 2 releases: 3 years (I'm optimistic and consider that
> developers pay attention to PendingDeprecationWarning). So deprecation
> during 3 releases with the new release cadence would fit the previous
> practice: 3 years ;-)
>
> Are you suggesting to start with a PendingDeprecationWarning? I recall
> a debate about the value of this warning which is quiet by default,
> whereas DeprecationWarning is now displayed in the __main__ module.
>
> I'm not sure about enforcing the removal. Some APIs were deprecated,
> but it was unsure if we really wanted to remove them. Or because we
> had no idea how long it would take for developers to upgrade to the
> new functions.
>
> For example, the C API for the old Unicode API using "Py_UNICODE*"
> type is deprecated since Python 3.3 but there is no concrete plan to
> remove it (just a vague "Python 4.0 removal" plan).
>
> I deprecated the support for bytes filenames on Windows in Python 3.3,
> but Steve Dower implemented his PEP 529 (use UTF-8 for bytes on
> Windows) in Python 3.6 and so the deprecation warnings have been
> simply removed!
>
> I would 

[Python-Dev] Re: Travis CI for backports not working.

[Kyle Stanley]

Steve Dower wrote:
> As a related aside, I've been getting GitHub Actions support together
> (which I started at the sprints).

Would adding support for GitHub Actions make it easier/faster to
temporarily disable and re-enable specific CI services when they're having
external issues? IIUC, that seems to be the primary concern to address.

Note that I'm not particularly well acquainted with GitHub Actions, other
than briefly looking over https://github.com/features/actions.

On Fri, Dec 6, 2019 at 1:01 PM Steve Dower  wrote:

> On 06Dec2019 0620, Victor Stinner wrote:
> > What's the status? Right now, I see Travis CI jobs passing on 3.7,
> > 3.8 and master branches so I don't understand the problem. Maybe the
> > issue has been fixed and Travis CI can be made mandatory again?
>
> They've been passing fine for me too, I'm not quite sure what the issue
> was.
>
> As a related aside, I've been getting GitHub Actions support together
> (which I started at the sprints). My test PR is at
> https://github.com/zooba/cpython/pull/7 if anyone wants to check out
> what it could look like. Feel free to leave comments there.
>
> (One of these days I'll have to join core-workflow I guess...)
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/MTD2EQC3BWEGEV3LNKPF3KEJBSQN24LS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PEHYMEYY2VGW5SKU6SR6RUTIQQ3C7JP4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Kyle Stanley]

> BTW, I think 2to3 can help to move from 2&3 code to 3-only code.

> Instead, we can do:

> * Don't recommend u-prefix except in Python 2&3 code.
> * Provide a tool to remove the u-prefix.

+1, this seems like the smoothest way of handling it and has very minimal
impact on users. In 5+ years from now as u-strings become increasingly
rare, we can consider a long term deprecation process. In the meantime, any
tools we can provide to automate the process of converting them will make
the transition that much easier.

On Wed, Dec 4, 2019 at 12:41 AM Inada Naoki  wrote:

> On Wed, Dec 4, 2019 at 11:49 AM Ned Batchelder 
> wrote:
> >
> > On 12/3/19 8:13 PM, Inada Naoki wrote:
> > > I think it is too early to determine when to remove it.
> > > Even only talking about it causes blaming war.
> >
> > Has anyone yet given a reason to remove it?
>
> Note that "never" is included in ”when".
> I didn't promoting removing it.  I just said let's stop discussion about
> it.
>
>
> > It will change working code
> > into broken code.  Why do that?
>
> Of course, everyone in this thread understands it.
> No one proposes remove it now.
>
> On the other hand, we remove some parts from Python language
> and the stdlib regularly to keep Python clean.
> All removal will break some code.  That's why we have a deprecation period.
>
> Currently, u-prefix is very widely used.  It shouldn't be removed anytime
> soon.
> And I agree that we shouldn't raise DeprecationWarning for now.
>
> But how about 5, 10, and 20 years later?  No one knows it.
> Let's stop discussing it.  It can not be productive.
>
> Instead, we can do:
>
> * Don't recommend u-prefix except in Python 2&3 code.
> * Provide a tool to remove the u-prefix.
>
> Regards,
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EVKCCO5KMOGEEFMSSY2PZRVGT2LDOB5K/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AKUXJR6BO4WTO7SLN3ES6PCWNO5H62ZU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Kyle Stanley]

  > The number of live coroutines in a running interpreter.

Could you further elaborate on what is meant by "live coroutines"? My
guesses (roughly from most likely to least likely) would be:

1) All known coroutine objects in a state of either CORO_RUNNING or
CORO_SUSPENDED, but *not* CORO_CREATED or CORO_CLOSED.
2) Coroutine objects that are currently running, in a state of CORO_RUNNING
3) Coroutines objects being awaited, in a state of CORO_SUSPENDED
4) All known coroutine objects in a state of either CORO_CREATED,
CORO_RUNNING, or CORO_SUSPENDED
5) All known coroutine objects in any state
6) Total declared coroutines

Just so we're all on the same page, I'm referring to a "coroutine object"
as the object returned from the call `coro()`; whereas a "coroutine" is the
coroutine function/method `async def coro` (or the deprecated
generator-based coroutines).

It probably wouldn't be as much of a concern to only allow 1M running
coroutines at one time, but it might be an issue to only allow for there to
be 1M known coroutine objects in any state within a given interpreter.
Particularly for servers that run nearly indefinitely and handle a
significant number of concurrent requests.

On Tue, Dec 3, 2019 at 11:24 AM Mark Shannon  wrote:

> Hi Everyone,
>
> I am proposing a new PEP, still in draft form, to impose a limit of one
> million on various aspects of Python programs, such as the lines of code
> per module.
>
> Any thoughts or feedback?
>
> The PEP:
> https://github.com/markshannon/peps/blob/one-million/pep-100.rst
>
> Cheers,
> Mark.
>
>
> Full text
> *
>
> PEP: 100
> Title: The one million limit
> Author: Mark Shannon 
> Status: Active
> Type: Enhancement
> Content-Type: text/x-rst
> Created: 03-Dec-2019
> Post-History:
>
>
>
> Abstract
> 
> This PR proposes a limit of one million (1 000 000) for various aspects
> of Python code and its implementation.
>
> The Python language does not specify limits for many of its features.
> Not having any limit to these values seems to enhance programmer freedom,
> at least superficially, but in practice the CPython VM and other Python
> virtual
> machines have implicit limits or are forced to assume that the limits are
> astronomical, which is expensive.
>
> This PR lists a number of features which are to have a limit of one
> million.
> If a language feature is not listed but appears unlimited and must be
> finite,
> for physical reasons if no other, then a limit of one million should be
> assumed.
>
>
> Motivation
> ==
>
> There are many values that need to be represented in a virtual machine.
> If no limit is specified for these values, then the representation must
> either be inefficient or vulnerable to overflow.
> The CPython virtual machine represents values like line numbers,
> stack offsets and instruction offsets by 32 bit values. This is
> inefficient, and potentially unsafe.
>
> It is inefficient as actual values rarely need more than a dozen or so
> bits to represent them.
>
> It is unsafe as malicious or poorly generated code could cause values to
> exceed 2\ :sup:`32`.
>
> For example, line numbers are represented by 32 bit values internally.
> This is inefficient, given that modules almost never exceed a few
> thousand lines.
> Despite being inefficent, is is still vulnerable to overflow as
> it is easy for an attacker to created a module with billions of newline
> characters.
>
> Memory access is usually a limiting factor in the performance of modern
> CPUs.
> Better packing of data structures enhances locality and reduces memory
> bandwith,
> at a modest increase in ALU usage (for shifting and masking).
> Being able to safely store important values in 20 bits would allow
> memory savings
> in several data structures including, but not limited to:
>
> * Frame objects
> * Object headers
> * Code objects
>
> There is also the potential for a more efficient instruction format,
> speeding up interpreter dispatch.
>
> Rationale
> =
>
> Imposing a limit on values such as lines of code in a module, and the
> number of local variables,
> has significant advantages for ease of implementation and efficiency of
> virtual machines.
> If the limit is sufficiently large, there is no adverse effect on users
> of the language.
>
> By selecting a fixed but large limit for these values,
> it is possible to have both safety and efficiency whilst causing no
> inconvience to human programmers
> and only very rare problems for code generators.
>
> One million
> ---
>
> The Java Virtual Machine (JVM) [1]_ specifies a limit of 2\ :sup:`16`-1
> (65535) for many program
> elements similar to those covered here.
> This limit enables limited values to fit in 16 bits, which is a very
> efficient machine representation.
> However, this limit is quite easily exceeded in practice by code
> generators and
> the author is aware of existing Python code that already exceeds 2\
> :sup:`16` lines of code.
>
> A limit of one million fits 

[Python-Dev] Re: Implementation of PEP-0604

[Kyle Stanley]

> I agree with Nick, this is indeed big, but not impossible. If you are not
sure yet whether you will work on implementation, you can focus on
polishing the PEP text, and then if it is accepted and you will decide to
give implementation to someone else, we will find a volunteer.

Unless I'm misunderstanding something, it seems that the OP is under the
impression that because they authored the PEP, they're obligated and
responsible for the entire implementation. Instead, we should be able to
relatively easily split up the implementation into multiple self-contained
PRs. The OP can still work on some of them if they want to and are able to,
but other contributors can work on other parts of it.

The easiest way of doing this would likely be separating each
self-contained PR into a bullet point in a bpo issue with a brief
description (similar to the layout posted above), and then the OP can work
through them as they have time to do so. Any other contributor can simply
pick up where the OP left off, or on other parts of it while the OP is
working on something else. This would of course be after/if the PEP is
accepted.

On Mon, Nov 4, 2019 at 7:33 AM Ivan Levkivskyi  wrote:

> On Thu, 31 Oct 2019 at 23:20, Nick Coghlan  wrote:
>
>> [...snip...]
>>
>>>
>>> It's not enough to move the typing classes, I must move
>>> functools.lru_cache() and dependencies, collections.abs.Mapping and
>>> dependencies, and track the frame level.
>>>
>>> *It's too big for me.*
>>>
>>>
>> It's certainly not an easy project to tackle.
>>
>> For some of the specific points you raise though, you wouldn't translate
>> the existing Python code directly to C.
>>
>
> I agree with Nick, this is indeed big, but not impossible. If you are not
> sure yet whether you will work on implementation, you can focus on
> polishing the PEP text, and then if it is accepted and you will decide to
> give implementation to someone else, we will find a volunteer.
>
> --
> Ivan
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/AKXBKSBYESEJVKAAPCXHG4Z5EWOS54MM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YYAKCVANFGRJAVGML4WJ225M5ANYZFDD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: A much better tokenize.untokenize function

[Kyle Stanley]

> This is all irrelevant if you haven't signed the CLA*.

IMO, this shouldn't be an argument against the proposed changes, as the CLA
is fairly straightforward and only takes 24-48 hours to process. Unless the
OP specifically is unable to or doesn't want to sign the CLA, it won't be a
significant concern. Most new authors usually open their first PR without
having the CLA already signed, then have the CLA signed and processed while
waiting for review from a core developer.

On Mon, Nov 4, 2019 at 11:48 AM Ethan Furman  wrote:

> On 11/04/2019 03:42 AM, Edward K. Ream wrote:
> > On Sun, Nov 3, 2019 at 7:23 PM Edward K. Ream wrote:
>
> > Let's be clear, there is no reason for python libraries to include every
> little code gem. However, I have two motivations adding the gem to the
> tokenize module:
>
> This is all irrelevant if you haven't signed the CLA*.
>
> --
> ~Ethan~
>
>
> * https://www.python.org/psf/contrib/contrib-form/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/F6MZYCE3FRHAIBFCSMVQ2JTKQDRO5TAJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3ZC5VFJGGJBJE467HQ5E3KX346EC6OEN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Allow custom headers in `http.server`

[Kyle Stanley]

> Probably python-list/comp.lang.python mailing list/news group is the
best place

Since this involves a potential suggested change (adding a feature to
assign custom headers to http.server), python-ideas or the "Ideas" section
of https://discuss.python.org/ would also be appropriate.

On Sat, Oct 26, 2019 at 11:05 AM Oleg Broytman  wrote:

> Hello.
>
>This mailing list is to work on developing Python (adding new
> features to Python itself and fixing bugs); if you're having problems
> learning, understanding or using Python, please find another forum.
> Probably python-list/comp.lang.python mailing list/news group is the
> best place; there are Python developers who participate in it; you may
> get a faster, and probably more complete, answer there. See
> https://www.python.org/community/ for other lists/news groups/fora.
> Thank you for understanding.
>
> On Sat, Oct 26, 2019 at 03:29:03PM +0300, Alex Yursha <
> alexyur...@gmail.com> wrote:
> > Hi CPython maintainers,
> >
> > I need to test my CORS setup and looking for a possibility to set a
> > custom *Access-Control-Allow-Origin
> > *header in http.server. As of now, there is no such feature. Are you
> > interested in me writing a patch to contribute a feature of setting
> custom
> > headers directly to `http.server`?
>
>You can override method ``send_headers`` of the class
> ``HTTPRequestHandler`` and add your own headers. See an example at
>
> https://docs.aws.amazon.com/polly/latest/dg/example-Python-server-code.html
>
> > Best,
> > - Alex
>
> Oleg.
> --
> Oleg Broytmanhttps://phdru.name/p...@phdru.name
>Programmers don't die, they just GOSUB without RETURN.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SSJL2BEQNII23ZGMLAXWYJYKTK35H6TO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EUCMXIWKK4ICGAIOZ457LD7QUEDU7TTB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Awareness creation of a new IDE

[Kyle Stanley]

> You don't need permission to support Python in a commercial IDE. It's
open source.

Adding to this, see
https://docs.python.org/3/license.html#terms-and-conditions-for-accessing-or-otherwise-using-python
for the complete licensing information on Python.

On a somewhat related note to the licensing page, I just noticed that the
link to Zope's website (http://www.zope.com) is no longer valid. They
currently use a .org TLD instead of .com (https://www.zope.org). Mind if I
open a PR to fix/update the link in Doc/licensing.rst?

On Mon, Oct 21, 2019 at 1:53 PM Guido van Rossum  wrote:

> Hi Lasan,
>
> You don't need permission to support Python in a commercial IDE. It's open
> source. If you use the Python logo, make sure it refers to Python and that
> you don't claim to be endorsed by the PSF, that's all we ask.
>
> Good luck!
>
> --Guido
>
> On Mon, Oct 21, 2019 at 9:41 AM Contact - Google 
> wrote:
>
>> Dear Sir / Madam ,
>>
>> I'm Lasan Nishshanka, CEO and the Founder of Clenontec. We are creating
>> an IDE. It's name is Bacend Studio. We're creating this for the Windows
>> Operating System. This IDE is designed to perform special functions such
>> as App Development, Software Development, Games Development, Machine
>> Learning, Artificial Intelligence etc.
>> We will send you this letter to get permission to add the PYTHON
>> Programming Language to our IDE. If you can give it permission, please
>> tell us. We await a speedy reply from you.
>>
>> Company Web Site : https://www.clenontec.com/
>>
>> E Mail : clenon...@gmail.com
>>
>> Thank You.
>>
>>
>> --
>> This email has been checked for viruses by AVG.
>> https://www.avg.com
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/LKL5DNJAGAQLNPVJF5DVGIBITZZ2DMBA/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OJ5DAQYXZXFUR6NF7BGRFNYQDY2CAW6R/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RTSYN4K3FD5WX4GANRMUB6QL6YZPAADF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help to fix test_asyncio issues

[Kyle Stanley]

> Recently, I started to experiment "./python -m test [options] -F
-j100" to attempt to reproduce some tricky race conditions: -j100
spawns 100 worker processes in parallel and -F stands for --forever
(run tests in loop and stop at the first failure).

Interesting, even as someone who has recently worked within
asyncio a decent amount recently, I was entirely unaware of
this method of producing race conditions in the tests. If continued
experimentation shows this to be a reliable means of reproducing
various race conditions, it might be worth adding this to
https://devguide.python.org/runtests/.

Thanks for sharing!


On Mon, Oct 21, 2019 at 7:08 AM Victor Stinner  wrote:

> Hi,
>
> Right now, there are 14 open issues with "test_asyncio" in the title.
> Many test_asyncio tests have race conditions. I'm trying to fix them
> one by one, but it takes time, and then new tests are added with new
> race condition :-( For example, the following new test is failing
> randomly on Windows:
>
> "Windows: test_asyncio: test_huge_content_recvinto() fails randomly
> with ProactorEventLoop" is failing randomly since 6 months:
> https://bugs.python.org/issue36732
>
> test_asyncio uses more and more functional tests which is a good
> thing. In the early days of asyncio, most tests mocked more than half
> of asyncio to really be "unit test". But at the end, the test tested
> more mocks than asyncio... The problem of functional tests is that
> it's hard to design them properly to avoid all race conditions,
> especially when you consider multiplatform (Windows, macOS, Linux,
> FreeBSD, etc.).
>
> It would help me if someone could try to investigate these issues,
> provide a reliable way to reproduce them, and propose a fix. (Simply
> saying that you can reproduce the test and that you would like to work
> on an issue doesn't really help, sorry.)
>
> Recently, I started to experiment "./python -m test [options] -F
> -j100" to attempt to reproduce some tricky race conditions: -j100
> spawns 100 worker processes in parallel and -F stands for --forever
> (run tests in loop and stop at the first failure). I was surprised
> that my Fedora 30 didn't burn in blame. In fact, the GNOME desktop
> remains responsible even with a system load higher than 100. The Linux
> kernel (5.2) is impressive! Under such high system load (my laptop has
> 8 logical CPUs), race conditions are way more likely.
>
> The problem of test_asyncio is that it's made of 2160 tests, see:
>
>./python -m test test_asyncio --list-cases
>
> You may want to only run a single test case (class) or even a single
> test method: see --match option which can be used multiple times to
> only run selected test classes or selected test methods. See also
> --matchfile which is similar but uses a file. Example:
>
> $ ./python -m test test_asyncio --list-cases > cases
> # edit cases
> $ ./python -m test test_asyncio --matchfile=cases
>
> test_asyncio is one of the most unstable test: I'm getting more and
> more buildbot-status emails about test_asyncio... likely because we
> fixed most of the other race conditions which is a good thing ;-)
>
> Some issues look to be specific to Windows, but it should be possible
> to reproduce most issues on Linux as Linux. Sometimes, it's just that
> some specific Windows buildbot workers are slower than other buildbot
> workers.
>
> Good luck ;-)
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/R7X6NKGEOKWD3PBWIL2LPZWZ6MMRANN5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7K4Z3QJTZCRALCIIFEJSQXVCMBYU37J2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Currently working on the release of Python 3.8.0rc1

[Kyle Stanley]

> Can you help with the "What's New" document?

After seeing this, I went through the whatsnew for 3.8 and found a couple
of typos and broken links (from Sphinx roles that were slightly off). I
opened a PR to fix as many as I was able to:
https://github.com/python/cpython/pull/16535. Raymond will be the one to
make the final decision as the editor, but I would appreciate it if anyone
could help with verifying the changes and providing feedback. I'd like to
have the PR finished a good bit before October 14th to avoid last minute
additions. Having additional reviewers would likely help to accelerate the
process.

On Mon, Sep 30, 2019 at 3:53 AM Łukasz Langa  wrote:

> Team,
> amazing job on getting us back on track over the weekend.
>
> *Thank you*
> All release blockers and deferred release blockers solved. And there was
> relatively little additional activity on the branch -- as expected at this
> point! Thank you for this, it will help get the release candidate out on
> time.
>
> *I'm working on cutting RC1 today*
> Hopefully all sanity checks, as well as building the source tarball and
> the binary installers for macOS and Windows will all work out fine and
> we'll be seeing RC1 out tonight.
>
> *RC2 and the date of 3.8.0 gold*
> If we manage to avoid the need for RC2, we will be able to release 3.8.0
> on October 14th. If we need an RC2, that will slip by a week. I hope we
> won't. Ideally RC1 should be identical codewise to 3.8.0.
>
> To help our chances, please avoid any source-related activity on the 3.8
> branch from now until the release of 3.8.0. Yes, that also counts for bug
> fixes unless they are critical. (Yeah, it might be a bit annoying but
> nobody wants to be the person who introduced a last minute regression in a
> major release.)
>
> Note I didn't say I forbid any activity. I have no power over you,
> actually. More importantly though, I trust your judgement if you assess
> some bug is bad enough the fix absolutely has to get into 3.8.0. Moreover,
> I specifically said source-related activity because...
>
> 3.8.0 is important. To some users it will be the first and the last
> release in the 3.8 series they will see.
>
> *We need you to focus on the docs now*
> Are all your changes properly documented?
> Did you notice other changes you know of to have insufficient
> documentation?
> Can you help with the "What's New" document?
>
> anxiously configure-&&-makingly y'rs
> - Ł
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ACMKEQNGK4FVUIZ6TYD5H26OSPIO5GSN/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YQ66MSEXD5PN5VVTIJA3E4U32MNIIMIL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Adding GitHub usernames to the developer log

[Kyle Stanley]

> devguide now uses developers.rst which is generated from the following
private file:
https://github.com/python/voters/blob/master/python-core.toml

Ah good to know. I don't have access to the voters repo so I wasn't sure
about the exact format or location of the file.

> Guessing a GitHub identifier from a real name is not really easy.

Yeah that's my main concern, particularly for new contributors trying to
find or identify a specific core developer.

For some people it's quite easy when it matches up with their real name
(such as yourself and Brett for example). But for others it's definitely
not obvious, especially when their real name isn't included in any part of
their username. I admit that I'm guilty of this myself with using
"aeros167" instead of "kstanley" or something along those lines (I had my
GitHub account for a decent while before I started contributing to Python).
This would be a non-issue with a public listing of core developer real
names -> GitHub usernames that isn't dependent on bpo.

On Wed, Sep 25, 2019 at 7:12 AM Victor Stinner  wrote:

> Le mer. 25 sept. 2019 à 08:24, Kyle Stanley  a écrit :
> > Recently, Brett updated the developer log in the devguide
> > (https://devguide.python.org/developers/) to fetch the names of each
> core
> > developer and the date they were given commit privileges from the private
> > python-committers repository.
>
> devguide now uses developers.rst which is generated from the following
> private file:
> https://github.com/python/voters/blob/master/python-core.toml
>
> The TOML file contains developer identifiers for:
>
> * GitHub
> * bugs.python.org (Roundup)
> * discuss.python.org (Discourse)
>
> > The motivation behind creating a a new topic for this issue was Brett's
> > response to my comment in the PR that updated the devguide
> > (https://github.com/python/devguide/pull/533#issuecomment-532405907).
> > Essentially, if no core developers have an issue with having their GitHub
> > username posted on the devguide, we can move forward with adding it.
>
> I'm in favor of making the GitHub identifiers public since it's part
> of the trust relationship between core developers and contributors.
> Some operations in our workflow *requires* a core developer on GitHub
> pull requests. So it's good to be able to check who are core
> developers on GitHub.
>
> Guessing a GitHub identifier from a real name is not really easy.
>
> The GitHub identifer can be *guessed* using the public bugs.python.org
> data. For example, I'm user 2377 on bugs.python.org which shows that
> my GitHub identifier is vstinner:
> https://bugs.python.org/user2377
>
> "Is Committer [hidden]" doesn't help to check if it's real or a fake
> account :-/ Some core developers have multiple bugs.python.org
> accounts.
>
> Note: I changed my bugs.python.org and GitHub identifiers one year ago
> from "haypo" to "vstinner" :-)
>
> It seems like https://github.com/orgs/python/teams/python-core/members
> is private.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NKDS3ELJH3N3ARWJH2FAMBJ7KX4J3JKC/

[Python-Dev] Re: Adding GitHub usernames to the developer log

[Kyle Stanley]

> But I will say that listing our GitHub usernames with our real names is
not required to tell who is a core developer. In GitHub's UI there are
multiple places it will tell you if a person is a member of the
repository/team

Within the GitHub UI if you're not a member of the organization, it tells
you who's a member of the org, but it does _not_ tell you what specific
team they're on. There are close to 100 people that are in the Python
organization on various teams but not a member of python-core. This
includes anyone on the more recently created python-triage team or any of
the PSF teams. I only became aware of this more recently, I'm not sure as
to whether or not it's a configurable setting that is modifiable by the
organization admins.

To verify this, simply log out of your GitHub account and go into a PR with
multiple organization members. Hovering over their name will show "Member
of Python" but not the actual name of the team(s) they're on. For a recent
example, logout of your GitHub account, go to PR-16375 (
https://github.com/python/cpython/pull/16375), then compare the hover cards
between myself and Yury. Many people have their role within the
organization mentioned in their GitHub bio (which appears on the hover
card), but this is not always the case.

It's also very useful when trying to find a specific core developer,
particularly when trying to locate the GitHub usernames for specific
experts. Locating the GitHub usernames of the experts is a bit of a
separate issue, but having the usernames listed on the developer log would
at least make it easier to do so without relying on bpo.

On Wed, Sep 25, 2019 at 12:48 PM Brett Cannon  wrote:

> I'm personally fine listing GItHub usernames in the devguide's developer
> list, but I'm also not trying to be anonymous on GitHub.
>
> But I will say that listing our GitHub usernames with our real names is
> not required to tell who is a core developer. In GitHub's UI there are
> multiple places it will tell you if a person is a member of the
> repository/team (e.g. hover cards, each comment a teammate makes).
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/KOKMLHX4RSCWPCUR3YEQBVRB6R3VJOVS/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SGKRKMWXOAUW26332QAK2456QLDWL3AP/

[Python-Dev] Adding GitHub usernames to the developer log

[Kyle Stanley]

Recently, Brett updated the developer log in the devguide
(https://devguide.python.org/developers/) to fetch the names of each core
developer and the date they were given commit privileges from the private
python-committers repository.

I think it would also be quite useful to include GitHub usernames on that list.
Currently, the only list that contributors can find the GitHub usernames for
each core developer is through the committers list on bpo. Since we will be
moving away from bpo (PEP 581), we should have a comprehensive list that is
separate from that platform.

The motivation behind creating a a new topic for this issue was Brett's
response to my comment in the PR that updated the devguide
(https://github.com/python/devguide/pull/533#issuecomment-532405907).
Essentially, if no core developers have an issue with having their GitHub
username posted on the devguide, we can move forward with adding it.


Another related but more long term project is adding the GitHub usernames
to the experts index (https://devguide.python.org/experts/). This is more
involved because the bpo nosy list currently pulls from the experts index,
meaning the nosy list is dependent on the specific formatting used.

To address this, I opened a PR a couple of months ago which would add a .json
file containing the data from the experts index
(https://github.com/python/devguide/pull/517), based on the discussion in the
related issue (https://github.com/python/devguide/issues/507). If any available
core developers are experienced with structuring .json files, I would greatly
appreciate any feedback.

The next step would be converting the nosy list script to use the new .json
file instead of the experts index page, so that we could adjust the page
to also include GitHub usernames. Optimally, the contents in the experts
index would be pulled from the .json file automatically so any changes only
have to be made to a single location.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FPCSKVI6DXYV5FKT7BJZXVVIXYN2M52Y/

[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?

[Kyle Stanley]

> How easy would it be to search the sources and the docs for the ones
that are currently public but not documented?

Comparing the docs and .c files for functions that are not documented
would be fairly trivial, but it's very difficult to define something as
being
public if it's not documented, since that's the main definition used. If
it's not
in the documentation, there's not a definitive way tell that it's public
(AFAIK).

The presence or lack of the name starting with an underscore can sometimes
be an indicator, but that's not consistently reliable.

On Mon, Sep 23, 2019 at 7:59 PM MRAB  wrote:

> On 2019-09-23 21:22, Vinay Sajip via Python-Dev wrote:
> > OK - but that's just one I picked at random. There are others like it -
> what would be the process for deciding which ones need to be made private
> and moved? Should an issue be raised to track this?
> >
> It's not really a question of which ones should be made private, but of
> which ones should be remain public. If it's mentioned in the docs, then
> it's public, otherwise it should be private.
> How easy would it be to search the sources and the docs for the ones
> that are currently public but not documented?
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4KUPJBNXXL7VD6JPAC7YCCTD56A5OUAZ/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U3WTOSLDLIZBIB4IJPJ4LSMWQNLY7W2V/

[Python-Dev] Re: The Python 2 death march

[Kyle Stanley]

Benjamin, what are you thoughts on usage of the "needs backport to 2.7"
label? For most of the PRs I've reviewed I tend to avoid adding it myself,
but I've seen it used periodically. It seems to be used rather infrequently
(
https://github.com/python/cpython/pulls?q=is%3Apr+label%3A%22needs+backport+to+2.7%22+is%3Amerged),
but I'm not entirely clear on when it should be used, particularly for
documentation-related PRs.

Also, is there an official date when the label will be removed? Removing
the backport label seems to generally be at the discretion of the release
manager for that version, but I was curious as to whether it would be
removed on the sunset date or prior to then. Without the label, backport
PRs can still be opened manually of course, but removing it would probably
help to discourage 2.7 backports.

On Mon, Sep 9, 2019 at 9:38 AM Benjamin Peterson 
wrote:

> Hi all,
> It's finally time to schedule the last releases in Python 2's life. There
> will be two more releases of Python 2.7: Python 2.7.17 and Python 2.7.18.
>
> Python 2.7.17 release candidate 1 will happen on October 5th followed by
> the final release on October 19th.
>
> I'm going to time Python 2.7.18 to coincide with PyCon 2020 in April, so
> attendees can enjoy some collective catharsis. We'll still say January 1st
> is the official EOL date.
>
> Thanks to Sumana Harihareswara, there's now a FAQ about the Python 2
> sunset on the website: https://www.python.org/doc/sunset-python-2/
>
> Regards,
> Benjamin
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/APWHFYQDKNVYQAK3HZMBGQIZHAVRHCV2/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NYIVL47NMGMSWTF5FQCZQF2ZWPA6I2SH/

[Python-Dev] Re: Python for Windows (python-3.7.4.exe) location confusing

[Kyle Stanley]

Terry Jan Reedy wrote:
> "This installer also installs the multiversion py.exe launcher if doing 
so would not be a downgrade.  If you have admin access, we recommend 
installing one py.exe for all users.  See xxx for how to use py.exe.
[ ] Install py.exe for all users.

+1, Particularly on this part. If the user installing Python has administrative 
access, I don't see much of a reason for them to have to install a separate 
launcher and installation. Recommending a global py.exe install for them seems 
to be far more straight forward. 

I'm not certain as to whether or not we are able to collect statistics on this, 
but I would not be at all surprised if the majority of users installing Python 
had administrative access on the device they're installing it on. While I think 
the ability to allow non-admin users to install packages in their own install 
is very useful, it should not come at the expense of making the process 
confusing for those with administrative privileges.

Steve Dower wrote:
> It also means 
that regular users can install packages without needing to be admin, and 
without corrupting other user's installs.

Does this have any advantage over using a virtual environment? I can imagine 
this might be more simple for new users (since they don't have to do anything 
additional), but setting up a virtual environment to use for a separate set of 
packages is very straightforward nowadays with venv.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WDQRQ5ZJDGABRQUKZ7C6WMIS5PEAAROA/

[Python-Dev] Re: Announcing the new Python triage team on GitHub

[Kyle Stanley]

> For instance, I don't think a triager should be able to edit other users'
comments or lock conversations, but I'm afraid GitHub
> doesn't provide that level of granularity (correct?).

You are correct in that GitHub does not allow customization of the
permissions, but those with the  "Triage" permission are unable to edit
comments or lock conversations. In order to do either of those, the "Write"
permission is required. Compared to a normal contributor, triagers are only
able to add labels, close, reopen, and assign PRs/Issues.

On Fri, Aug 23, 2019 at 2:01 PM Giampaolo Rodola' 
wrote:

> If the concern is pruning old/stale/invalid issues/PRs, I think the
> proposal to add a specific "awaiting-close" (or similar) label should work.
> These issues can periodically be handled by a core-dev who can click on the
> label, then evaluate and bulk-close them in one shot if necessary. Also, it
> seems to me that including "devguide" and "core-workflow" repos is
> unnecessary since they are low-traffic administrative repos related to our
> internal workflow, they are attended by a small portion of the core-dev
> team only and labels don't play a central role as in the cpython repo.
> Please let's keep in mind that this role implies higher responsibilities
> than a usual contributor, and since the bar for being promoted is
> considerably lower than a core-dev's, triager permissions should reflect
> that and remain minimal. For instance, I don't think a triager should be
> able to edit other users' comments or lock conversations, but I'm afraid
> GitHub doesn't provide that level of granularity (correct?).
>
> --
> Giampaolo - http://grodola.blogspot.com
>
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2YYB6MAJ55U2U4KTKINFDSB4USJ3NMZY/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZR6STQ3FCSKPS5BRJRTVEZ637NDDYTX5/