[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Jeremy Kloth]

On Tue, Mar 29, 2022 at 11:55 AM Brett Cannon  wrote:
> You're right that is the fundamental problem. But for me this somewhat stems 
> from the fact that we don't have a shared understanding of what the stdlib 
> is,  and so the stdlib is a bit unbounded in its size and scope. That leads 
> to a stdlib which is hard to maintain. It's just like dealing with any scarce 
> resource: you try to cut back on your overall use as best as you can and then 
> become more efficient with what you must still consume; I personally think we 
> don't have an answer to the "must consume" part of that sentence that leads 
> us to "cut back" to a size we can actually keep maintained so we don't have 
> 1.6K open PRs.

This sent me down a rabbit hole of trying to analyze the PRs to see
where things are failing.  Using a couple libraries from PyPI
(pygithub and gitignore_parser) I've mined out some interesting
things.  Some of the lines in CODEOWNERS are not matching what people
think they are matching so notifications are being missed.  The
"partial matches" are PRs which have at least one matching file in
CODEOWNERS so I think that those people should have been notified as
well as those where the entire changeset matches via CODEOWNERS.

I'm still fine tuning the script, so if more detail is desired (or
some other ignorable entries like /Misc/NEWS.d/ or /**/clinic/*.h)
please give feedback.

So with that, here are some results:

== PR Totals 
core developers: 281
triage team: 89
draft PRs: 37
CODEOWNERS: 139
partial matches: 653
orphans: 396
-
Total PRs: 1595

Of the files changed in all of the PRs, the following that do not
match via CODEOWNERS:
-- Core (/Grammar/, /Include/, /Objects/, /Python/) that should always
be maintained
Include/Python.h
Include/bltinmodule.h
Include/boolobject.h
Include/buffer.h
Include/bytes_methods.h
Include/ceval.h
Include/cpython/code.h
Include/cpython/dictobject.h
Include/cpython/fileutils.h
Include/cpython/object.h
Include/cpython/pyerrors.h
Include/cpython/pystate.h
Include/cpython/pythonrun.h
Include/cpython/sysmodule.h
Include/descrobject.h
Include/exports.h
Include/internal/pycore_atomic.h
Include/internal/pycore_bitutils.h
Include/internal/pycore_dict.h
Include/internal/pycore_format.h
Include/internal/pycore_global_strings.h
Include/internal/pycore_interp.h
Include/internal/pycore_object.h
Include/internal/pycore_pathconfig.h
Include/internal/pycore_pylifecycle.h
Include/internal/pycore_runtime.h
Include/internal/pycore_runtime_init.h
Include/internal/pycore_sysmodule.h
Include/methodobject.h
Include/modsupport.h
Include/moduleobject.h
Include/object.h
Include/obmalloc.h
Include/opcode.h
Include/opcode_names.h
Include/py_curses.h
Include/pydtrace.h
Include/pyerrors.h
Include/pymath.h
Include/pyport.h
Include/pyportosf.h
Include/pystate.h
Include/pystrtod.h
Include/pythread.h
Include/structmember.h
Objects/abstract.c
Objects/boolobject.c
Objects/bytearrayobject.c
Objects/bytes_methods.c
Objects/bytesobject.c
Objects/classobject.c
Objects/complexobject.c
Objects/descrobject.c
Objects/fileobject.c
Objects/funcobject.c
Objects/genericaliasobject.c
Objects/listobject.c
Objects/longobject.c
Objects/memoryobject.c
Objects/methodobject.c
Objects/moduleobject.c
Objects/namespaceobject.c
Objects/object.c
Objects/obmalloc.c
Objects/picklebufobject.c
Objects/rangeobject.c
Objects/sliceobject.c
Objects/stringlib/split.h
Objects/structseq.c
Objects/tupleobject.c
Objects/unicodectype.c
Objects/unicodeobject.c
Objects/unicodetype_db.h
Objects/unionobject.c
Objects/weakrefobject.c
Python/Python-ast.c
Python/_warnings.c
Python/bltinmodule.c
Python/ceval_gil.h
Python/codecs.c
Python/condvar.h
Python/dtoa.c
Python/errors.c
Python/fileutils.c
Python/formatter_unicode.c
Python/getargs.c
Python/hashtable.c
Python/importlib.h
Python/importlib_external.h
Python/initconfig.c
Python/makeopcodetargets.py
Python/modsupport.c
Python/opcode_targets.h
Python/pathconfig.c
Python/preconfig.c
Python/pylifecycle.c
Python/pystate.c
Python/pystrtod.c
Python/structmember.c
Python/symtable.c
Python/sysmodule.c
Python/thread_nt.h
Python/thread_pthread.h

-- Truly Orphaned
.azure-pipelines/ci.yml
.azure-pipelines/libffi-build.yml
.azure-pipelines/pr
.azure-pipelines/pr.yml
.azure-pipelines/windows-steps.yml
.github/workflows/build.yml
.github/workflows/build_msi.yml
.github/workflows/deploy-previews.yml
.github/workflows/doc.yml
.github/workflows/verify-bundled-wheels.yml
Doc/c-api/arg.rst
Doc/c-api/dict.rst
Doc/c-api/init.rst
Doc/c-api/intro.rst
Doc/c-api/long.rst
Doc/c-api/object.rst
Doc/c-api/structures.rst
Doc/c-api/type.rst
Doc/c-api/typeobj.rst
Doc/c-api/unicode.rst
Doc/conf.py
Doc/data/refcounts.dat
Doc/extending/embedding.rst
Doc/extending/extending.rst
Doc/extending/newtypes.rst
Doc/extending/newtypes_tutorial.rst
Doc/extending/windows.rst
Doc/glossary.rst
Doc/howto/functional.rst
Doc/includes/custom.c
Doc/includes/custom2.c
Doc/includes/custom3.c
Doc/includes/custom4.c
Doc/includes/sqlite3/blob.py

[Python-Dev] Re: (no subject)

[Ethan Furman]

[woops]
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NAT7FGQUCSCB2I265P7IR5BATLKHL5FT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] (no subject)

[Stanislav Smolec 15.03-1983]

RANK
SLOBODNIK

NAME
STANISLAV

FRST NAME
SMOLEC

BINARY
15.03/1983

IDENTIFICATION NUMBER
830315/6081

ADDRESS
KUČIŠDORFSKÁ DOLINA

NUMBER HOME
6144/704

CITY
PEZINOK

COUNTRY
SLOVAKIA

ZIP CODES
90201

EUROPE

COMPANY 4KA SLOVAKIA
MY CALL NUMBER
+421949281444

COMPANY MICROSOFT
MY E-MAIL
stanislav.smo...@outlook.sk

COMPANY GOOGLE
MY E-MAIL
it.technik.slova...@gmail.com

ACCOUNT BANK 365.bank a. s.
SK83 6500   9681 1747

CODES ID CARD
IDSVKHW852826<48303156081<
8303154M2909303SVK<<<3
SMOLEC

[Python-Dev] Re: Enhancing generic type documentation in the standard library

[Guido van Rossum]

On Wed, Mar 30, 2022 at 12:42 PM Steve Holden  wrote:

> Not defining the semantics of annotations was a brave move, but
> inevitably led to the situation where different use cases, all
> quite reasonable, would spring into being. Now they have, the development
> team has to decide a) which ones to sanction and b) which will be left out
> in the cold.
>

Interesting, in the JavaScript (excuse me, ECMAScript) world, there's
currently a proposal on the table to do just that, on steroids:
https://github.com/giltayar/proposal-types-as-comments (full disclosure:
some of the proposal authors are colleagues of mine). We'll have to see how
they fare.


> I wish them well.
>
> Kind regards,
> Steve
>
>
> On Wed, Mar 30, 2022 at 5:24 PM Christopher Barker 
> wrote:
>
>> > I personally would love for a typing.python.org or equivalent
>> subsection of docs.python.org to exist.
>>
>> +1
>>
>> There’s a wrinkle here, however. The type specs are Python, but how they
>> are used/interpreted is up to third party packages.
>>
>> So it’s a bit tricky to know exactly what to document where.
>>
>> I don’t think that’s insurmountable, but something to keep in mind.
>>
>> For example, while the clear specs are the first step, what the community
>> really could use is a good “recommended practices for static typing” doc —
>> and that’s harder to do without reference to particular tools.
>>
>> -CHB
>>
>>
>> --
>> Christopher Barker, PhD (Chris)
>>
>> Python Language Consulting
>>   - Teaching
>>   - Scientific Software Development
>>   - Desktop GUI and Web Development
>>   - wxPython, numpy, scipy, Cython
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/YF5UIQIWNVIANFOCIQ2J4DJACQGJDGVM/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NW3SRUA2H3KNZBD4CICXY5QF2YWELUV5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YUGMANOPKSAR7SPDLQGTDGFAID6H5YHA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enhancing generic type documentation in the standard library

[Steve Holden]

Not defining the semantics of annotations was a brave move, but
inevitably led to the situation where different use cases, all
quite reasonable, would spring into being. Now they have, the development
team has to decide a) which ones to sanction and b) which will be left out
in the cold.

I wish them well.

Kind regards,
Steve


On Wed, Mar 30, 2022 at 5:24 PM Christopher Barker 
wrote:

> > I personally would love for a typing.python.org or equivalent
> subsection of docs.python.org to exist.
>
> +1
>
> There’s a wrinkle here, however. The type specs are Python, but how they
> are used/interpreted is up to third party packages.
>
> So it’s a bit tricky to know exactly what to document where.
>
> I don’t think that’s insurmountable, but something to keep in mind.
>
> For example, while the clear specs are the first step, what the community
> really could use is a good “recommended practices for static typing” doc —
> and that’s harder to do without reference to particular tools.
>
> -CHB
>
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/YF5UIQIWNVIANFOCIQ2J4DJACQGJDGVM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NW3SRUA2H3KNZBD4CICXY5QF2YWELUV5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Sebastian Berg]

On Wed, 2022-03-30 at 17:51 +0200, Petr Viktorin wrote:
> On 30. 03. 22 17:42, Guido van Rossum wrote:
> > In the not so distant past I have proposed to introduce a new
> > category, 
> > "Unstable APIs". These are public but are not guaranteed to be
> > backwards 
> > compatible in feature releases (though I feel they should remain so
> > in 
> > bugfix releases).
> > 
> > I'm not sure whether those should have a leading underscore or not.
> > Perhaps (like some other languages do and like maybe we've used in
> > a few 
> > places) the name could just include the word "Unstable"?
> 
> IMO, the underscore should mark an API as internal: it can change at
> any 
> time (though in practice it often doesn't, e.g. to accommodate
> projects 
> that used it before a policy is written down).
> 

That is fair, although there are documented underscored ones:
https://docs.python.org/3/search.html?q=_Py

I suppose that means all bets are off _unless_ it is documented or
later adopted as stable API (e.g. `_PyObject_Vectorcall`).

With that, the only "not obviously OK" use in NumPy that I am aware of
is `_Py_HashDouble` (it seems undocumented).

Maybe "unless documented" is just a clear enough distinction in
practice.
Although, to some degree, I think it would be clearer if symbols that
have a realistic chance of changing in bug-fix releases had an
additional safe-guard.

- Sebastian



> This is useful e.g. for macros/static functions that wrap access to 
> something private, where the definition needs to be available but
> marked 
> "keep off".
> 
> 
> > On Wed, Mar 30, 2022 at 8:08 AM Victor Stinner
> >  > > wrote:
> > 
> >     The internal C API can be used on purpose. But there is no
> > backward
> >     compatibility warranty and it can change anytime. In practice,
> > usually
> >     it only changes in 3.x.0 releases. For example, these private C
> > API
> >     changed in Python 3.9 and Python 3.11 (see my first email in
> > the other
> >     PEP 523 thread).
> > 
> >     To use the internal C API, you have to declare the
> > Py_BUILD_CORE macro
> >     and include an internal C API header file. For
> >     _PyInterpreterState_SetEvalFrameFunc(), it should be:
> > 
> >     #ifndef Py_BUILD_CORE_MODULE
> >     #  define Py_BUILD_CORE_MODULE
> >     #endif
> >     #include 
> >     #include  //
> >     _PyInterpreterState_SetEvalFrameFunc()
> >     #include   // _PyEval_EvalFrameDefault
> > 
> >     Victor
> > 
> >     On Tue, Mar 29, 2022 at 12:26 AM Jason Ansel via Python-Dev
> >     mailto:python-dev@python.org>> wrote:
> >  >
> >  > The PyTorch team plans to use PEP 523 as a part of PyTorch
> > 2.0,
> >     so this proposal may break the next major release of PyTorch.
> >  >
> >  > The related project is TorchDynamo, which can be found here:
> >  > https://github.com/facebookresearch/torchdynamo
> >     
> >  >
> >  > We will likely move this into the core of PyTorch closer to
> > release.
> >  >
> >  > If the changed happens, would PyTorch still be able to use
> > the
> >     eval frame API?  Or would it prevent from being used entirely?
> >  > ___
> >  > Python-Dev mailing list -- python-dev@python.org
> >     
> >  > To unsubscribe send an email to python-dev-le...@python.org
> >     
> >  >
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> >     
> >  > Message archived at
> >    
> > https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/
> >    
> >  > e/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/>
> >  > Code of Conduct: http://python.org/psf/codeofconduct/
> >     
> > 
> > 
> > 
> >     --
> >     Night gathers, and now my watch begins. It shall not end until
> > my death.
> >     ___
> >     Python-Dev mailing list -- python-dev@python.org
> >     
> >     To unsubscribe send an email to python-dev-le...@python.org
> >     
> >     https://mail.python.org/mailman3/lists/python-dev.python.org/
> >     
> >     Message archived at
> >    
> > https://mail.python.org/archives/list/python-dev@python.org/message/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/
> >    
> >  > e/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/>
> >     Code of Conduct: http://python.org/psf/codeofconduct/
> >     
> > 
> > 
> > 
> > -- 
> > --Guido van Rossum (python.org/~guido )
> > 

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Skip Montanaro]

On Wed, Mar 30, 2022, 12:02 PM Toshio Kuratomi  wrote:

>
> As just one example, i found two interesting items in the discussion
> started by Skip about determining what modules don't have maintainers just
> downstream if this.
>

Age in snake years doesn't necessarily correlate well with one's desire to
take a deep dive into the documentation. Just sayin'...  These
answers might have been there waiting for a more diligent search on my part.

Skip

>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QRBIANZOIJHGKKQVNBZ7PHECGTFHBPBI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Guido van Rossum]

I don't think such a guarantee (to not vary internal APIs from 3.x.y to
3.x.(y+1)) has ever before been made in writing, although in practice we've
been doing this (more so now than we were in the 2.x timeframe).

I think we should not lightly vary internal APIs between bugfix releases,
but at the same time I am not sure I want to absolutely guarantee that no
internal API ever changes in a bugfix release (because that might prevent
fixing certain bugs).

So I think we need to officially embrace a category of "unstable public
APIs" and set a policy specifically for those, before we can make progress.

I'd like the SC to take some initiative here.

On Wed, Mar 30, 2022 at 9:34 AM Jason Ansel via Python-Dev <
python-dev@python.org> wrote:

> Got it, thanks for the clarifications!
>
> Tracking 3.x Python versions is fine.  We already need to do that to
> support things like new bytecodes, and PyTorch supports an explicit list of
> 3.x Python versions with separate builds for each.
>
> Tracking 3.x.y Python versions would be much more painful, and make us
> need to rethink our approach.
>
> So what Steve Downer described as "remain compatible within a single 3.x
> release", seems like exactly what we want.  I support that level of
> compatibility guarantee.  Could we keep that guarantee with this change?
>
> Thanks,
> Jason
>
>
>
>
>
> 
> From: Victor Stinner 
> Sent: Wednesday, March 30, 2022 7:56 AM
> To: Steve Dower
> Cc: Jason Ansel; python-dev@python.org
> Subject: Re: [Python-Dev] Re: C API: Move PEP 523 "Adding a frame
> evaluation API to CPython" private C API to the internal C API
>
> On Wed, Mar 30, 2022 at 2:26 AM Steve Dower 
> wrote:
> > Right now, the API is allowed/expected to change between 3.x releases
> > (which normally we don't allow without a deprecation period) but it
> > still has to remain compatible within a single 3.x release. Making it
> > fully internal *without adding a stability guarantee* means it could
> > change more frequently, which you wouldn't be able to handle as an
> > installable package.
> >
> > It's *unlikely* that it'll change that often, because there are still
> > other public interfaces that cannot. But, the plan behind this is to
> > make more stuff internal so that it can be modified more freely, so we
> > may see that rate of change increase.
>
> Well, my plan is not break these internal C API just for fun. It's
> only to better "advertise" the separation between the "stable" public
> C API (backward compatibility warranty) and the "unstable"
> private/internal C API (can change any time, changes are not
> documented).
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/U7M65SSDZMM7LNAKEDZZ4KKQIFTCOQ2M/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J3G5RO4FUO3Y3WJ2AJAUMJNS2BXYZ7J7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Fwd: Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Sasha Kacanski]

-- Forwarded message -
From: Sasha Kacanski 
Date: Wed, Mar 30, 2022, 2:55 PM
Subject: Re: [Python-Dev] Re: Are "Batteries Included" still a Good Thing?
[was: It's now time to deprecate the stdlib urllib module]
To: Paul Moore 
Cc: Christopher Barker , Ethan Furman <
et...@stoneleaf.us>, Python Dev 


Agree with Paul,
Best of python always was, you can do almost everything with standard libs,
but then novice becomes better developer and starts exploring libs and
tools that wrap stuff or perform better or has functionality that stdlib
does not support right of the bet they start using variety of libs that
suits their needs.
Regards,
Sasha


On Sun, Mar 27, 2022, 8:06 PM Paul Moore  wrote:

> On Sun, 27 Mar 2022 at 17:11, Christopher Barker 
> wrote:
> >
> > With the json package included, all they need to do is `import json`. If
> that wasn't there, they's look in PyPi for a JSON implementation, and find
> an absolutely astonishing number of options. I just did a search for "JSON"
> >  on PYPI, and it's HUGE -- most of them are for specialized JSON-using
> protocols of some sort. I was actually really surprised that couple I know
> about of the top of my head (ujson, orjson) are actually hard to find.
> >
> > "You can just pip install it" is really not a good solution.
> >
> > In fact, this is an example, I think, of where we should put some effort
> into making the included batteries better -- it's great to have a JSON lib
> built in, but it's too bad that it's not best-of-bread by pretty much any
> definition (speed, memory performance, flexibility) -- there are quite a
> few feature requests open for it -- it would be nice to actually implement
> some of those. (but yes, that's a lot of work that someone(s) would have to
> do)
> >
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it (as is httpx, and urllib3, and aiohttp). But
> urllib is for people who want to get a file from the web, and *not*
> have to deal with dependencies, 3rd party libraries, etc.
>
> The "batteries included" standard library and PyPI complement each
> other. Neither is redundant, and neither implies the other is
> unnecessary.
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/E2FGXTYOEYLC6GSJGMWBQKHOO62LZPHQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3LOIPKKY73KDH526SKLXJX4FGBCEQMLH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Brett Cannon]

On Wed, Mar 30, 2022 at 4:53 AM Barney Gale  wrote:

> On Wed, 30 Mar 2022 at 12:20, Antoine Pitrou  wrote:
>
>> On Wed, 30 Mar 2022 12:03:58 +0100
>> Steve Dower  wrote:
>> > On 30Mar2022 1124, Barney Gale wrote:
>> > > I'd like to become a maintainer for the pathlib module, if possible.
>> I
>> > > know the code and tests inside-out, and I've been wrestling the
>> > > internals for past few Python releases. I check the bugs/PRs at least
>> > > every week and help wherever I can.
>> >
>> > Antoine is still active, so if he's fine with that, I'm also
>> supportive.
>> > You're certainly familiar enough with that module.
>>
>> I'm entirely supportive. Huge thank you for working on pathlib!
>>
>
> Thank you Antoine, Steve. It's humbling to stand on your shoulders and
> thrilling to give back to Python. If I'm successful in my application I'll
> do my best to take good care of pathlib and hopefully urllib too.
>

The first step is to get triage permissions:
https://devguide.python.org/triaging/?highlight=triage#python-triage-team .
We can't add anyone to the CODEOWNERS file unless you have write
permissions, which means becoming a core dev. I have added myself to
CODEOWNERS for pathlib and I will try to remember to @ mention you on
things, Barney (https://github.com/python/cpython/pull/32202).

Antoine, I left you out of my PR for CODEOWNERS just because I thought it
was presumptuous to add anyone to more GitHub notifications without their
permission.


>
> These batteries ain't dead yet! :D
>

Definitely not all of them. 

-Brett


>
> Best,
>
> Barney
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RWVLBPUYNCLCJUYKFR2GWU3V5IT55UCG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NR2WVMXF64G7KK6IBGBGYNTKX2AEZUMK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Toshio Kuratomi]

On Tue, Mar 29, 2022, 10:55 AM Brett Cannon  wrote:

>
>
> On Tue, Mar 29, 2022 at 8:58 AM Ronald Oussoren 
> wrote:
>
>>
>>
>> On 29 Mar 2022, at 00:34, Brett Cannon  wrote:
>>
>>
>>
>> On Mon, Mar 28, 2022 at 11:52 AM Christopher Barker 
>> wrote:
>>
>>> On Mon, Mar 28, 2022 at 11:29 AM Paul Moore  wrote:
>>>


>> Having such a policy is a good thing and helps in evolving the stdlib,
>> but I wonder if the lack of such a document is the real problem.   IMHO the
>> main problem is that the CPython team is very small and therefore has
>> little bandwidth for maintaining, let alone evolving, large parts of the
>> stdlib.  In that it doesn’t help that some parts of the stdlib have APIs
>> that make it hard to make modifications (such as distutils where
>> effectively everything is part of the public API).  Shrinking the stdlib
>> helps in the maintenance burden, but feels as a partial solution.
>>
>
> You're right that is the fundamental problem. But for me this somewhat
> stems from the fact that we don't have a shared understanding of what the
> stdlib *is*,  and so the stdlib is a bit unbounded in its size and scope.
> That leads to a stdlib which is hard to maintain. It's just like dealing
> with any scarce resource: you try to cut back on your overall use as best
> as you can and then become more efficient with what you must still consume;
> I personally think we don't have an answer to the "must consume" part of
> that sentence that leads us to "cut back" to a size we can actually keep
> maintained so we don't have 1.6K open PRs
> .
>

One of the things that's often missed in discussions is that a *good*
policy document can also help grow the number of maintainers.

As just one example, i found two interesting items in the discussion
started by Skip about determining what modules don't have maintainers just
downstream if this. (1) There's a file which matches maintainers to modules
in the stdlib (this is documented but i only found out about it a few years
ago and Skip, who's been around even longer than me didn't know about it
either... So this means something about how our policy docs are currently
structured could be improved).  (2) Terry brought up that you don't have to
be a core maintainer in order to take up ownership of something in the
stdlib. That's awesome!  But this is definitely something i didn't know.
I've been "focusing"[1] on  becoming a core maintainer because i thought
that was a prerequisite to getting anything done in the stdlib. Knowing
that getting involved with stdlib maintenance is different could be vastly
helpful.

[1] focusing is the wrong word... It expresses the feeling of "directed
action" correctly but doesn't convey the lack of activity that sprinkles my
attempts.  Nor does it account for discouragement, helplessness, and
imposter-y feelings which are the reasons for that lack.

-toshio
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GBNDUQXWTBGCP5243L4HUU5UVLKQ7UWB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Jason Ansel via Python-Dev]

Got it, thanks for the clarifications!

Tracking 3.x Python versions is fine.  We already need to do that to support 
things like new bytecodes, and PyTorch supports an explicit list of 3.x Python 
versions with separate builds for each.

Tracking 3.x.y Python versions would be much more painful, and make us need to 
rethink our approach.

So what Steve Downer described as "remain compatible within a single 3.x 
release", seems like exactly what we want.  I support that level of 
compatibility guarantee.  Could we keep that guarantee with this change?

Thanks,
Jason






From: Victor Stinner 
Sent: Wednesday, March 30, 2022 7:56 AM
To: Steve Dower
Cc: Jason Ansel; python-dev@python.org
Subject: Re: [Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation 
API to CPython" private C API to the internal C API

On Wed, Mar 30, 2022 at 2:26 AM Steve Dower  wrote:
> Right now, the API is allowed/expected to change between 3.x releases
> (which normally we don't allow without a deprecation period) but it
> still has to remain compatible within a single 3.x release. Making it
> fully internal *without adding a stability guarantee* means it could
> change more frequently, which you wouldn't be able to handle as an
> installable package.
>
> It's *unlikely* that it'll change that often, because there are still
> other public interfaces that cannot. But, the plan behind this is to
> make more stuff internal so that it can be modified more freely, so we
> may see that rate of change increase.

Well, my plan is not break these internal C API just for fun. It's
only to better "advertise" the separation between the "stable" public
C API (backward compatibility warranty) and the "unstable"
private/internal C API (can change any time, changes are not
documented).

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U7M65SSDZMM7LNAKEDZZ4KKQIFTCOQ2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Gyro Funch]

On 3/30/2022 12:32 PM, Baptiste Carvello wrote:

Le 28/03/2022 à 18:44, Steve Dower a écrit :

I think to most people "batteries included" doesn't necessarily mean
"standard library," with all that implies. It just means "it came with
the first thing that I installed" (or alternatively, "at no additional
charge").


A point I have not seen made, is that some uses really need *standard*
batteries.

In scientific research, you'll sometimes share a data-processing script
among a large group of non-computer-specialists, who hopefully all have
*some* Python+NumPy installed: it may be a full up-to-date Conda, or it
may be "a very old version that a former colleague installed for me
years ago and now I won't update it for fear I break it". You have to
cater to all versions.

It may be more glamorous to compete with Rust and JS for "best modern
language", than with Excel for "lowest common denominator for
calculation". Still, that's a use case where Python has historically
been strong, and it's one of the reasons it works so well in the
research world.

Hopefully this use case is not forgotten, even if those
non-computer-specialists users are less likely to be involved here in
core development.

Cheers,
Baptiste



This is a use case in which I am very interested. Reproducibility is 
extremely important in my field as well. However, I don't think that any 
solution offered by the standard python distribution will serve this 
need, while still allowing the language to evolve.
The conda virtual environment mechanism, with carefully specified 
package versions is a useful approach, but does not cover differences in 
results owing to the user's OS, architecture, libraries, etc.
Among the alternatives, solutions based on containers are probably the 
best bet when reproducibility is essential.


-gyro
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2DGEWKHKKSHSYYFJBRJUO7VLUMOW456Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enhancing generic type documentation in the standard library

[Christopher Barker]

> I personally would love for a typing.python.org or equivalent subsection
of docs.python.org to exist.

+1

There’s a wrinkle here, however. The type specs are Python, but how they
are used/interpreted is up to third party packages.

So it’s a bit tricky to know exactly what to document where.

I don’t think that’s insurmountable, but something to keep in mind.

For example, while the clear specs are the first step, what the community
really could use is a good “recommended practices for static typing” doc —
and that’s harder to do without reference to particular tools.

-CHB


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YF5UIQIWNVIANFOCIQ2J4DJACQGJDGVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[John Ehresman]

As someone who maintains a debugger that uses private api’s, I’d like to see 
some commitment to seeing them not change in micro releases such as 3.11.1 -> 
3.11.2. Micro releases should be compatible with other micro releases of the 
same major.minor release such as 3.11 so that an extension compiled against 
3.11.1 will work with 3.11.2.

Compatibility was broken (at least) once in a 2.x series (maybe 2.4?) and we 
started getting reports of people using 2.x.0 were having problems. 
Fortunately, in that case compiling against 2.x.0 produced an extension that 
worked against all micro releases.

Note that I’m not asking for private api’s not change in major.micro releases 
such as 3.10 -> 3.11. I know that there can be very good reasons to change 
private api's because that I probably will have work to do in order to support 
a new major.micro release.

I also don’t think that exposing everything that every extension needs with a 
non-private api is a good idea because then the internals will be more 
difficult to change — you’d be signing up to support that api for all future 
major.micro versions until there’s a compatibility break.

John

> On Mar 30, 2022, at 11:01 AM, Victor Stinner  wrote:
> 
> The internal C API can be used on purpose. But there is no backward
> compatibility warranty and it can change anytime. In practice, usually
> it only changes in 3.x.0 releases. For example, these private C API
> changed in Python 3.9 and Python 3.11 (see my first email in the other
> PEP 523 thread).
> 
> To use the internal C API, you have to declare the Py_BUILD_CORE macro
> and include an internal C API header file. For
> _PyInterpreterState_SetEvalFrameFunc(), it should be:
> 
> #ifndef Py_BUILD_CORE_MODULE
> #  define Py_BUILD_CORE_MODULE
> #endif
> #include 
> #include  // _PyInterpreterState_SetEvalFrameFunc()
> #include   // _PyEval_EvalFrameDefault
> 
> Victor
> 
> On Tue, Mar 29, 2022 at 12:26 AM Jason Ansel via Python-Dev
>  wrote:
>> 
>> The PyTorch team plans to use PEP 523 as a part of PyTorch 2.0, so this 
>> proposal may break the next major release of PyTorch.
>> 
>> The related project is TorchDynamo, which can be found here:
>> https://github.com/facebookresearch/torchdynamo
>> 
>> We will likely move this into the core of PyTorch closer to release.
>> 
>> If the changed happens, would PyTorch still be able to use the eval frame 
>> API?  Or would it prevent from being used entirely?
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at 
>> https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TK27HEVDUWQQZQ4XIKTQLDHWK52ZLHWN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Steve Dower]

On 3/30/2022 4:42 PM, Guido van Rossum wrote:
In the not so distant past I have proposed to introduce a new category, 
"Unstable APIs". These are public but are not guaranteed to be backwards 
compatible in feature releases (though I feel they should remain so in 
bugfix releases).


Agreed. This is definitely a new category, and it seems the only thing 
we're debating now is whether or not to add/omit the underscore.


I'm not sure whether those should have a leading underscore or not. 
Perhaps (like some other languages do and like maybe we've used in a few 
places) the name could just include the word "Unstable"?


I don't think we have "Unstable" anywhere, though we do have "Unsafe" 
(which is more about threading than stability, so not right for this). 
But I'd be okay with that as a compromise.


I'd prefer to not have public-unstable APIs hidden behind the same 
preprocessor directive as internal APIs. That's a big switch to throw 
that may also activate other settings - for example, on Windows we will 
set the minimum Windows version in our headers if you enable internal 
APIs, and disable automatic linking of the Python DLL. Easy enough 
things to work around, but they probably need to be explicitly 
documented as well if we're going to document public APIs as requiring 
Py_BUILD_CORE (and I don't want to have to document that kind of stuff).


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZEPJSIJNXQKXTOE2MRNS6GJRP52WLDEF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Petr Viktorin]

On 30. 03. 22 17:42, Guido van Rossum wrote:
In the not so distant past I have proposed to introduce a new category, 
"Unstable APIs". These are public but are not guaranteed to be backwards 
compatible in feature releases (though I feel they should remain so in 
bugfix releases).


I'm not sure whether those should have a leading underscore or not. 
Perhaps (like some other languages do and like maybe we've used in a few 
places) the name could just include the word "Unstable"?


IMO, the underscore should mark an API as internal: it can change at any 
time (though in practice it often doesn't, e.g. to accommodate projects 
that used it before a policy is written down).


This is useful e.g. for macros/static functions that wrap access to 
something private, where the definition needs to be available but marked 
"keep off".



On Wed, Mar 30, 2022 at 8:08 AM Victor Stinner > wrote:


The internal C API can be used on purpose. But there is no backward
compatibility warranty and it can change anytime. In practice, usually
it only changes in 3.x.0 releases. For example, these private C API
changed in Python 3.9 and Python 3.11 (see my first email in the other
PEP 523 thread).

To use the internal C API, you have to declare the Py_BUILD_CORE macro
and include an internal C API header file. For
_PyInterpreterState_SetEvalFrameFunc(), it should be:

#ifndef Py_BUILD_CORE_MODULE
#  define Py_BUILD_CORE_MODULE
#endif
#include 
#include  //
_PyInterpreterState_SetEvalFrameFunc()
#include   // _PyEval_EvalFrameDefault

Victor

On Tue, Mar 29, 2022 at 12:26 AM Jason Ansel via Python-Dev
mailto:python-dev@python.org>> wrote:
 >
 > The PyTorch team plans to use PEP 523 as a part of PyTorch 2.0,
so this proposal may break the next major release of PyTorch.
 >
 > The related project is TorchDynamo, which can be found here:
 > https://github.com/facebookresearch/torchdynamo

 >
 > We will likely move this into the core of PyTorch closer to release.
 >
 > If the changed happens, would PyTorch still be able to use the
eval frame API?  Or would it prevent from being used entirely?
 > ___
 > Python-Dev mailing list -- python-dev@python.org

 > To unsubscribe send an email to python-dev-le...@python.org

 > https://mail.python.org/mailman3/lists/python-dev.python.org/

 > Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/


 > Code of Conduct: http://python.org/psf/codeofconduct/




--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/


Code of Conduct: http://python.org/psf/codeofconduct/




--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EIK3XLPV7A7WVB4FA47PM2G2SJ42RERO/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MBHN5AR5C7S3SP3OBSFV3ES26PDM746R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Guido van Rossum]

In the not so distant past I have proposed to introduce a new category,
"Unstable APIs". These are public but are not guaranteed to be backwards
compatible in feature releases (though I feel they should remain so in
bugfix releases).

I'm not sure whether those should have a leading underscore or not. Perhaps
(like some other languages do and like maybe we've used in a few places)
the name could just include the word "Unstable"?

On Wed, Mar 30, 2022 at 8:08 AM Victor Stinner  wrote:

> The internal C API can be used on purpose. But there is no backward
> compatibility warranty and it can change anytime. In practice, usually
> it only changes in 3.x.0 releases. For example, these private C API
> changed in Python 3.9 and Python 3.11 (see my first email in the other
> PEP 523 thread).
>
> To use the internal C API, you have to declare the Py_BUILD_CORE macro
> and include an internal C API header file. For
> _PyInterpreterState_SetEvalFrameFunc(), it should be:
>
> #ifndef Py_BUILD_CORE_MODULE
> #  define Py_BUILD_CORE_MODULE
> #endif
> #include 
> #include  //
> _PyInterpreterState_SetEvalFrameFunc()
> #include   // _PyEval_EvalFrameDefault
>
> Victor
>
> On Tue, Mar 29, 2022 at 12:26 AM Jason Ansel via Python-Dev
>  wrote:
> >
> > The PyTorch team plans to use PEP 523 as a part of PyTorch 2.0, so this
> proposal may break the next major release of PyTorch.
> >
> > The related project is TorchDynamo, which can be found here:
> > https://github.com/facebookresearch/torchdynamo
> >
> > We will likely move this into the core of PyTorch closer to release.
> >
> > If the changed happens, would PyTorch still be able to use the eval
> frame API?  Or would it prevent from being used entirely?
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EIK3XLPV7A7WVB4FA47PM2G2SJ42RERO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Victor Stinner]

The internal C API can be used on purpose. But there is no backward
compatibility warranty and it can change anytime. In practice, usually
it only changes in 3.x.0 releases. For example, these private C API
changed in Python 3.9 and Python 3.11 (see my first email in the other
PEP 523 thread).

To use the internal C API, you have to declare the Py_BUILD_CORE macro
and include an internal C API header file. For
_PyInterpreterState_SetEvalFrameFunc(), it should be:

#ifndef Py_BUILD_CORE_MODULE
#  define Py_BUILD_CORE_MODULE
#endif
#include 
#include  // _PyInterpreterState_SetEvalFrameFunc()
#include   // _PyEval_EvalFrameDefault

Victor

On Tue, Mar 29, 2022 at 12:26 AM Jason Ansel via Python-Dev
 wrote:
>
> The PyTorch team plans to use PEP 523 as a part of PyTorch 2.0, so this 
> proposal may break the next major release of PyTorch.
>
> The related project is TorchDynamo, which can be found here:
> https://github.com/facebookresearch/torchdynamo
>
> We will likely move this into the core of PyTorch closer to release.
>
> If the changed happens, would PyTorch still be able to use the eval frame 
> API?  Or would it prevent from being used entirely?
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/
> Code of Conduct: http://python.org/psf/codeofconduct/



--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OQTAF6CQRKHQPYUY5HWVOTUAEXKHI5WE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Victor Stinner]

On Wed, Mar 30, 2022 at 2:26 AM Steve Dower  wrote:
> Right now, the API is allowed/expected to change between 3.x releases
> (which normally we don't allow without a deprecation period) but it
> still has to remain compatible within a single 3.x release. Making it
> fully internal *without adding a stability guarantee* means it could
> change more frequently, which you wouldn't be able to handle as an
> installable package.
>
> It's *unlikely* that it'll change that often, because there are still
> other public interfaces that cannot. But, the plan behind this is to
> make more stuff internal so that it can be modified more freely, so we
> may see that rate of change increase.

Well, my plan is not break these internal C API just for fun. It's
only to better "advertise" the separation between the "stable" public
C API (backward compatibility warranty) and the "unstable"
private/internal C API (can change any time, changes are not
documented).

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OMGUI5N33BG3EU4OG3IUZXJQF6XU7X2B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Nick Coghlan]

On Tue, 29 Mar 2022, 9:44 am Toshio Kuratomi,  wrote:

> One thing about talking about "make urllib more like requests" that is
> different than any of the other libs, though, is that requests aims to be
> easier to use than anything else (which I note Chris Barker called out as
> why he wanted urllib to be more like it).  I think that's important to
> think about because i think ease of use is also the number one thing that
> the audience you talk about is also looking for.
>
> Of course, figuring out whether an api like request's is actually easier
> to use than urllib or merely more featureful is open to debate.
>

There's a concrete scope difference between the two:

* requests was written as a way to make HTTP(S) requests from Python,
including authenticated JSON based API requests
* urllib was designed to work with a variety of URL schemas which may or
may not support authentication and encryption (before the rise of the web
and restrictive corporate firewalls helped HTTP become the de facto
standard for data exchange over the public internet)


I wrote more on that topic back in 2016 [1], and the gist of that article
still holds true: urllib is OK for the basic HTTP GET use case, but having
to tell urllib that you want to use HTTP-only features exposes a lot of low
level plumbing that end users ideally wouldn't have to care about.

Unfortunately, it isn't clear where a useful subset of a full-fledged HTTP
client library lies. "Make it easy to send a TLS protected JSON API POST
request with Basic authentication and check the status code of the
response" *might* qualify, but even that isn't particularly easy to define
or maintain (What about proxies? What about redirects? What about other
authentication methods?).

Thus the status quo persists - adding a fully featured HTTP client library
to the existing stdlib maintenance burden doesn't (or at least shouldn't)
strike *anyone* as a good idea, but it also isn't clear how much of the
ease of use of the existing 3rd party libraries comes from the sheer amount
of functionality they have built-in so users don't have to worry about it
themselves, which means defining something "lighter and easier to maintain
than a fully featured HTTP client library, but still easier to use than
calling in to urllib directly" isn't a viable option either.

At least with urllib present in the stdlib, it's *possible* to share "No
dependency" recipes for crafting particular HTTP requests, even though the
default recommendation remains to grab a higher level 3rd party library
instead of rolling your own solution.

Cheers,
Nick.


[1]
https://www.curiousefficiency.org/posts/2016/08/what-problem-does-it-solve.html






> -toshio
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4ZQC4H7HD3UXFT3CONU64YPOQBSPUTVY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QENBMBYL4YABJKRPX7UR65DOGRAZUJ26/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enhancing generic type documentation in the standard library

[Nick Coghlan]

On Wed, 30 Mar 2022, 3:49 am Brett Cannon,  wrote:

>
>
> On Mon, Mar 28, 2022 at 3:58 PM Luciano Ramalho 
> wrote:
>
>>
>> Documenting the generic types in the standard library is a much
>> smaller task than turning the typing PEPs into specs. It seems like a
>> good next step on the way to better typing docs.
>>
>
> Since the typing-sig will likely be doing the work and driving such an
> effort I would ask over there. I personally would love for a
> typing.python.org or equivalent subsection of docs.python.org to exist.
>

The import system gained its own subsection of the language reference to
get info out of the historical PEPs.

Perhaps gradual typing could go the same way, with the initial content
being:

* the type hinting docs for the builtin generics
* pointers to the PEPs that provide the interim docs until the reference
section is fleshed out

(we used the latter trick to bootstrap the PyPA specs page without having
to rewrite all the content: some of the pages are just pointers to the
relevant PEP)

Cheers,
Nick.


> -Brett
>
>
>>
>> Cheers,
>>
>> Luciano
>>
>>
>>
>> >
>> > -Brett
>> >
>> >>
>> >>
>> >> We now have lots of generic types in the standard library, but their
>> >> formal type parameters are poorly documented—or not documented at
>> >> all—in the standard library documentation.
>> >>
>> >> More importantly: the documentation we have about specific
>> >> covariant/contravariant types is now in entries in the `typing` module
>> >> that are all deprecated since PEP 585 was implemented in Python 3.9.
>> >>
>> >> Below I present two of many examples where the documentation of a
>> >> generic type is not great.
>> >>
>> >> However, if people agree this is a problem, we need to discuss where
>> >> and how to put the documentation in a way that is not too disruptive
>> >> to users of Python who don't know or don't care about type hints, for
>> >> many reasons that we should not judge.
>> >>
>> >> For example, where do we document the fact that `dict` accepts two
>> >> invariant formal type parameters, and that `frozenset` accepts one
>> >> contravariant type parameter?
>> >>
>> >> What do you think?
>> >>
>> >> Cheers,
>> >>
>> >> Luciano
>> >>
>> >> _
>> >> EXAMPLE 1: `Callable` variance is not documented
>> >>
>> >> For example, in the `Callable` type, the `ReturnType` parameter is
>> >> covariant, and the `ParameterType` parameters are all contravariant.
>> >>
>> >> But there is no mention of variance in this entry:
>> >>
>> https://docs.python.org/3/library/typing.html?highlight=typing#typing.Callable
>> >>
>> >> Also, no mention of the fact that `collections.abc.Callable` is
>> generic here:
>> >>
>> https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable
>> >>
>> >> PEP 483—The Theory of Type Hints—is the only official Python doc where
>> >> I found the information about the variance of the formal type
>> >> parameters of `Callable`. The explanation there is brilliant [0].
>> >>
>> >> [0] https://peps.python.org/pep-0483/#covariance-and-contravariance
>> >>
>> >> Regardless, the intended audience of PEPs is "core developers"—which
>> >> is neither a superset nor a subset of "Python devs now using type
>> >> hints". We should not rely on PEPs to document features for Python
>> >> users in general.
>> >>
>> >> _
>> >> EXAMPLE 2: `Generator` variance could be better documented
>> >>
>> >> The entry for `typing.Generator` [1] has this heading:
>> >>
>> >> class typing.Generator(Iterator[T_co], Generic[T_co, T_contra, V_co])
>> >>
>> >> Answer quickly: how many formal type parameters does `Generator`
>> >> require? Which are covariant? Which are contravariant?
>> >>
>> >> [1]
>> https://docs.python.org/3/library/typing.html?highlight=typing#typing.Generator
>> >>
>> >> Nowhere in that page [1] there's an explanation of the `*_co` and
>> >> `*_contra` naming convention, much less their semantics.
>> >>
>> >> Fortunately, the text of the `typing.Generator` entry says: "A
>> >> generator can be annotated by the generic type `Generator[YieldType,
>> >> SendType, ReturnType]".
>> >>
>> >> Unfortunately, `typing.Generator` is deprecated and will be gone in 5
>> >> years or so...
>> >>
>> >> The same issue applies to all the other generic types: builtins
>> >> (`dict`, `frozenset`), ABCs, etc.
>> >> Their formal type parameters they accept as generics are either
>> >> undocumented, or documented in parts of the `typing` module that are
>> >> already deprecated.
>> >>
>> >> Thoughts?
>> >>
>> >> --
>> >> Luciano Ramalho
>> >> |  Author of Fluent Python (O'Reilly, 2015)
>> >> | http://shop.oreilly.com/product/0636920032519.do
>> >> |  Technical Principal at ThoughtWorks
>> >> |  Twitter: @ramalhoorg
>> >> ___
>> >> Python-Dev mailing list -- python-dev@python.org
>> >> To unsubscribe send an email to python-dev-le...@python.org
>> >> 

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Paul Moore]

On Wed, 30 Mar 2022 at 12:28, Steve Dower  wrote:
>
> On 30Mar2022 1132, Baptiste Carvello wrote:
> > Le 28/03/2022 à 18:44, Steve Dower a écrit :
> >> I think to most people "batteries included" doesn't necessarily mean
> >> "standard library," with all that implies. It just means "it came with
> >> the first thing that I installed" (or alternatively, "at no additional
> >> charge").
> >
> > A point I have not seen made, is that some uses really need *standard*
> > batteries.
>
> I've certainly not forgotten it, it's just every time I try to make the
> point it doesn't seem to help. So until I find a clear way to argue
> that, yes, a standard "datetime" value makes sense, but no, a standard
> "HTTP headers dictionary" is unnecessary, I'm avoiding bringing it up
> again ;)

I also strongly agree with this position, and do my best to argue it.
But as you say, it is hard to draw the line.

Here's some examples, which while not exhaustive, might give a flavour
of my views:

"Obviously" things that should be in the stdlib (basic data types or
algorithms):
re, datetime, collections, enum, dataclasses, decimal, fractions
(maybe borderline), statistics, itertools, functools, pathlib

Obvious stuff that's OS related:
os, io, threading, multiprocessing, concurrent.futures,
subprocess, asyncio, urllib.request

Core services:
sys, venv, sysconfig, warnings, atexit, gc, zipimport, runpy,
importlib, marshal, pickle

Standard development tools:
argparse, pdb, cprofile, typing (arguable in some ways)

Handling of key file and data formats:
struct, csv, json, tomllib, zlib, gzip, bz2 (borderline), zipfile,
tarfile, base64

Arguable cases of file formats, but with important use cases
xml (you may not like it, but it's still used a lot)
configparser (similarly, ini-style files are used a lot, for
better or worse)
email (this is an outlier, but the packaging metadata file format
uses it, so there's a bootstrapping issue if it's not in the stdlib)

I've missed a lot out, and I'm sure people will argue most of these,
but that's sort of the point - it's *hard* to come up with a blanket
statement on how we should approach "the stdlib", because there's so
much diversity in there. That's why my starting position is that we
should retain the stdlib - sure, we can remove (and add!) stuff, but
we should do so on a case by case basis, not based on some blanket
"minimise it because we don't have the manpower or we don't want to
support it" basis.

If writing and sharing small, one-off scripts that depended on
non-stdlib packages were *significantly* easier, I might revise my
position on some of the arguable cases. PEP 582 (Python local packages
directory) might make a big difference here. But at the moment, adding
a 3rd party dependency effectively alters what you're doing from
"writing a script I can share" to "developing an application" - and
that's a *big* step for many people. Quoting Nathaniel Smith from
https://mail.python.org/archives/list/distutils-...@python.org/message/YFJITQB37MZOPOFJJF3OAQOY4TOAFXYM/
(albeit in a slightly different context):

"""
This last point is important: as projects grow in complexity, and are
used by a wider audience, they often end up with fairly complex tool
specifications that have to be shared among a team.
"""

My argument is that relying only on the stdlib allows people putting
off the need to take that step up in complexity, and we shouldn't
dismiss how significant that is.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LYKBB6YVZYQTJHPVRR5YMWE5D6YNSENY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Barney Gale]

On Wed, 30 Mar 2022 at 12:20, Antoine Pitrou  wrote:

> On Wed, 30 Mar 2022 12:03:58 +0100
> Steve Dower  wrote:
> > On 30Mar2022 1124, Barney Gale wrote:
> > > I'd like to become a maintainer for the pathlib module, if possible. I
> > > know the code and tests inside-out, and I've been wrestling the
> > > internals for past few Python releases. I check the bugs/PRs at least
> > > every week and help wherever I can.
> >
> > Antoine is still active, so if he's fine with that, I'm also supportive.
> > You're certainly familiar enough with that module.
>
> I'm entirely supportive. Huge thank you for working on pathlib!
>

Thank you Antoine, Steve. It's humbling to stand on your shoulders and
thrilling to give back to Python. If I'm successful in my application I'll
do my best to take good care of pathlib and hopefully urllib too.

These batteries ain't dead yet! :D

Best,

Barney
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RWVLBPUYNCLCJUYKFR2GWU3V5IT55UCG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Steve Dower]

On 30Mar2022 1132, Baptiste Carvello wrote:

Le 28/03/2022 à 18:44, Steve Dower a écrit :

I think to most people "batteries included" doesn't necessarily mean
"standard library," with all that implies. It just means "it came with
the first thing that I installed" (or alternatively, "at no additional
charge").


A point I have not seen made, is that some uses really need *standard*
batteries.


I've certainly not forgotten it, it's just every time I try to make the 
point it doesn't seem to help. So until I find a clear way to argue 
that, yes, a standard "datetime" value makes sense, but no, a standard 
"HTTP headers dictionary" is unnecessary, I'm avoiding bringing it up 
again ;)


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OCY7VGDIXJAQ55XAHOCYFZOBF6O4P4I4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Antoine Pitrou]

On Wed, 30 Mar 2022 12:03:58 +0100
Steve Dower  wrote:
> On 30Mar2022 1124, Barney Gale wrote:
> > I'd like to become a maintainer for the pathlib module, if possible. I 
> > know the code and tests inside-out, and I've been wrestling the 
> > internals for past few Python releases. I check the bugs/PRs at least 
> > every week and help wherever I can.  
> 
> Antoine is still active, so if he's fine with that, I'm also supportive. 
> You're certainly familiar enough with that module.

I'm entirely supportive. Huge thank you for working on pathlib!

Regards

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6N7YKZVCV6TLFIMWIUVRKFMJY34JUHYJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Baptiste Carvello]

Le 28/03/2022 à 18:44, Steve Dower a écrit :
> I think to most people "batteries included" doesn't necessarily mean
> "standard library," with all that implies. It just means "it came with
> the first thing that I installed" (or alternatively, "at no additional
> charge").

A point I have not seen made, is that some uses really need *standard*
batteries.

In scientific research, you'll sometimes share a data-processing script
among a large group of non-computer-specialists, who hopefully all have
*some* Python+NumPy installed: it may be a full up-to-date Conda, or it
may be "a very old version that a former colleague installed for me
years ago and now I won't update it for fear I break it". You have to
cater to all versions.

It may be more glamorous to compete with Rust and JS for "best modern
language", than with Excel for "lowest common denominator for
calculation". Still, that's a use case where Python has historically
been strong, and it's one of the reasons it works so well in the
research world.

Hopefully this use case is not forgotten, even if those
non-computer-specialists users are less likely to be involved here in
core development.

Cheers,
Baptiste
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V4QX4ZSHSUUMVQR3CRAEMBBR6N3WGVLM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Steve Dower]

On 30Mar2022 1124, Barney Gale wrote:
I'd like to become a maintainer for the pathlib module, if possible. I 
know the code and tests inside-out, and I've been wrestling the 
internals for past few Python releases. I check the bugs/PRs at least 
every week and help wherever I can.


Antoine is still active, so if he's fine with that, I'm also supportive. 
You're certainly familiar enough with that module.


I can also look after urllib if it would help. I have much less 
experience with that module vs pathlib, so I'll need to spend some time 
familiarizing myself with the code and the outstanding bug reports. But 
I do have plenty of experience with networking, HTTP, URLs, etc :-). 
I'll spend some time reading the bug tracker to start.


Please! I skim the open issues occasionally, but definitely feel out of 
my depth to decide on issues (e.g. should 
urlparse("http://[::1]spam:80;) raise? - I have literally no idea :) ), 
but feel free to nosy me on issues you think you've figured out and I 
can help confirm your logic and do merges. Or if another core dev has 
been helping you, keep nosying them.


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DRHN22EUHGYCEUHJ3GSMEYMM2BQCS3QR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Barney Gale]

On Mon, 28 Mar 2022 at 20:37, Steve Dower  wrote:

> email doesn't fix bugs; maintainers fix bugs. Please let us know
> *publicly* if you want to become the maintainer for a stdlib module and
> then we can support them, but if nobody is willing/able/ready to care
> for them it's irresponsible for us to keep shipping them to users.


I'd like to become a maintainer for the pathlib module, if possible. I know
the code and tests inside-out, and I've been wrestling the internals for
past few Python releases. I check the bugs/PRs at least every week and help
wherever I can.

I can also look after urllib if it would help. I have much less experience
with that module vs pathlib, so I'll need to spend some time familiarizing
myself with the code and the outstanding bug reports. But I do have plenty
of experience with networking, HTTP, URLs, etc :-). I'll spend some time
reading the bug tracker to start.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BRIL545G5KMWYJRJ327YPFOTO4IJONMP/
Code of Conduct: http://python.org/psf/codeofconduct/