[Python-Dev] Re: [Python-ideas] Re: Re: Have virtual environments led to neglect of the actual environment?

[Random832]

On Wed, Feb 24, 2021, at 09:08, Paul Moore wrote:
> I don't use Linux much, and I'm definitely not familiar with Linux
> distribution tools, but from what I can gather Linux distributions
> have made the choices:
> 
> 1. Write key operating system utilities in Python.
> 2. Share the Python interpreter and libraries.
> 3. Expose that Python interpreter as the *user's* default Python.

I think 1 *partially* mischaracterizes the problem, because any "system python" 
would logically be used by *every application written in python [or that embeds 
python] distributed by the OS's package management*, not just by "key operating 
system utilities". To suggest otherwise implies that they should not distribute 
any python applications at all.

That also makes asking all of their package maintainers to change their #! line 
to point at a different interpreter [or to pass an option, as I had suggested 
in another post] a more significant ask than the "just change a few core 
utilities" that some people seem to be assuming it would be. It also means that 
making a "system python" does not remove the need to distribute the largish 
subset of python *libraries* that they distribute, because even when these 
libraries aren't used by key OS utilities, they are still used by packaged 
applications.

[this, in turn, means we don't want the user's default python environment to 
stand entirely separate from the system python, or we'll start getting 
complaints along the lines of "I apt-get installed numpy, why can't I import it 
in my python interpreter?" - particularly from users who don't really care if 
it runs a couple versions behind]
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F7YNRI7LWU2VLS6AU6PQZUHHA3M5WFQJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Random832]

On Wed, Feb 24, 2021, at 06:27, Christian Heimes wrote:
> Separate directories don't prevent clashes and system breakage. But they
> provide an easy way to *recover* from a broken system.

I think it could be turned into a way to prevent them by A) having 
site-packages always take precedence over dist-packages [i believe this is 
already the case] in normal usage and B) providing an option to the 
interpreter, used by system scripts, to exclude site-packages entirely from the 
path.

Basically, site-packages would effectively be layered on top of "Lib + 
dist-packages" in a similar way to how a venv is layered on top of the main 
python installation - the inverse of the suggestion someone else in the thread 
made for the system python to be a venv. This wouldn't *exactly* be a venv 
because it wouldn't imply the other things that entering a venv does such as 
"python" [and script names such as pip] being an alias for the correct version 
of python, but it would provide the same kind of one-way isolation, whereby the 
"system environment" can influence the "normal environment" and not vice-versa, 
in the same way that packages installed in the main installation affect a venv 
[unless system-site-packages is disabled] but the venv obviously has no effect 
on the main installation.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K3DB2MSBWPDEK3O4UZ5VEQVKW7BFZDRU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Have virtual environments led to neglect of the actual environment?

[Random832]

I was reading a discussion thread 
 about various 
issues with the Debian packaged version of Python, and the following statement 
stood out for me as shocking:

Christian Heimes wrote:
> Core dev and PyPA has spent a lot of effort in promoting venv because we 
> don't want users to break their operating system with sudo pip install.

I don't think sudo pip install should break the operating system. And I think 
if it does, that problem should be solved rather than merely advising users 
against using it. And why is it, anyway, that distributions whose package 
managers can't coexist with pip-installed packages don't ever seem to get the 
same amount of flak for "damaging python's brand" as Debian is getting from 
some of the people in the discussion thread? Why is it that this community is 
resigned to recommending a workaround when distributions decide the 
site-packages directory belongs to their package manager rather than pip, 
instead of bringing the same amount of fiery condemnation of that practice as 
we apparently have for *checks notes* splitting parts of the stdlib into 
optional packages? Why demand that pip be present if we're not going to demand 
that it works properly?

I think that installing packages into the actual python installation, both via 
distribution packaging tools and pip [and using both simultaneously - the 
Debian model of separated dist-packages and site-packages folders seems like a 
reasonable solution to this problem] can and should be a supported paradigm, 
and that virtual environments [or more extreme measures such as shipping an 
entire python installation as part of an application's deployment] should 
ideally be reserved for the rare corner cases where that doesn't work for some 
reason.

How is it that virtual environments have become so indispensable, that no-one 
considers installing libraries centrally to be a viable model anymore? Are 
library maintainers making breaking changes too frequently, reasoning that if 
someone needs the old version they can just venv it? Is there some other cause?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KMRNKSVEPHAXA7BHFT4PWX4EKWYUF4G7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why aren't we allowing the use of C11?

[Random832]

On Thu, Jan 28, 2021, at 22:57, Emily Bowman wrote:
>  
> On Thu, Jan 28, 2021 at 1:31 PM MRAB  wrote:
> > I have Microsoft Visual Studio Community 2019 Version 16.8.4 (it's free) 
> > and it supports C11 and C17. 
> 
> While an upgrade for Community is free, for Pro/Enterprise without an 
> annual license it's not.

Something that's hard for me to find a clear answer for in a few minutes of 
searching:

Does the current version of the *Windows SDK* still come with a compiler, and 
can it be used without the Visual Studio IDE [the situation is confusing 
because it seems to install through the visual studio installer now] or at 
least without a license for Visual Studio Pro/Enterprise [i.e. by users who do 
not qualify for community edition] or with earlier versions of the Visual 
Studio IDE?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LUOB5H2S6PTHLZEDJVFUZTXLETS54KNX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

[Random832]

On Mon, Jan 25, 2021, at 18:44, Chris Jerdonek wrote:
> But to issue a warning when a standard module is being overridden like 
> I was suggesting, wouldn’t you also need to know whether the name of 
> the module being imported is a standard name, which is what 
> says.module_names provides?

I don't think the warning would be only useful for stdlib modules... has any 
thought been given to warning when a module being imported from the current 
directory / script directory is the same as an installed package?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OG7EUYVNNLPZIXRAX23DPK6WIWRKUX5D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: os.scandir bug in Windows?

[Random832]

On Fri, Oct 23, 2020, at 02:14, Random832 wrote:
> What correction, exactly, do you mean? The post I saw with the word 
> "Correction" on it is the one that *makes* the claim people are taking 
> issue with.

okay, sorry, I see the other correction post now...

My issue I guess was the same as Eryk Sun, it wasn't clear which parts of the 
previous post you were correcting and which (if any) you stood by, since they 
were about the behavior of different parts of the system, so it didn't register 
as a correction to that part when I originally read it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U4MZFDDMM4L52DKA6NBB7MKRJJ7QWEOB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: os.scandir bug in Windows?

[Random832]

On Tue, Oct 20, 2020, at 07:42, Steve Dower wrote:
> On 20Oct2020 0520, Rob Cliffe wrote:
> > On 19/10/2020 12:42, Steve Dower wrote:
> >> On 15Oct2020 2239, Rob Cliffe via Python-Dev wrote:
> >>> TLDR: In os.scandir directory entries, atime is always a copy of 
> >>> mtime rather than the actual access time.
> >>
> >> Correction - os.stat() updates the access time to _now_, while 
> >> os.scandir() returns the last access time without updating it.
> >>
> >> Eryk replied with a deeper explanation of the cause, but fundamentally 
> >> this is what you are seeing.
> >>
> >> Feel free to file a bug, but we'll likely only add a vague note to the 
> >> docs about how Windows works here rather than changing anything. If 
> >> anything, we should probably fix os.stat() to avoid updating the 
> >> access time so that both functions behave the same, but that might be 
> >> too complicated.
> >>
> >> Cheers,
> >> Steve
> > Sorry - what you say does not match the behaviour I observe, which is that
> 
> Yes, I posted a correction already (immediately after sending the first 
> email).

ok, see, the correction you posted doesn't address the part of your claim that 
people are taking issue with, which is that *calling os.stat() causes the atime 
to be set to the time of the call to os.stat()*. This is not the same thing as 
[correctly] saying that "calling os.stat() may return a more up-to-date atime, 
the time of the last read, write, or other operation", and the phrasing 
"updates the access time to _now_" certainly *seemed* unambiguous.

And at this point it's not clear to me whether you understand that people are 
reading your claim this way.

What correction, exactly, do you mean? The post I saw with the word 
"Correction" on it is the one that *makes* the claim people are taking issue 
with.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O63FMQYOHASHZ33CWBYQMD3H3XYGT5QC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 638: Syntactic macros

[Random832]

On Fri, Oct 16, 2020, at 18:59, Dan Stromberg wrote:
> The complexity of a language varies with the square of its feature 
> count,

Says who? I'd assume the orthogonality and regularity of features matters at 
least as much if not more than the number of features, and providing a system 
like this would guarantee some degree of regularity.

Is there some notion of "complexity of a language" [other than by trivially 
*defining* it as the square of the number of features] for which this can be 
shown to be true?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JVJDRCNNSDMDIIFWJ65MBQVTWC375SJW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: os.scandir bug in Windows?

[Random832]

On Mon, Oct 19, 2020, at 07:42, Steve Dower wrote:
> On 15Oct2020 2239, Rob Cliffe via Python-Dev wrote:
> > TLDR: In os.scandir directory entries, atime is always a copy of mtime 
> > rather than the actual access time.
> 
> Correction - os.stat() updates the access time to _now_, while 
> os.scandir() returns the last access time without updating it.

This is surprising - do we know why this happens?

Also, it doesn't seem true on my system with python 3.8.5 [and, yes, I checked 
that last access update is enabled for my test and updates normally when 
reading the file's contents].
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GX3KD4UQKJONCLOZY743WXNGENXL7YG2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we still need a support of non-type objects in issubclass()?

[Random832]

On Sat, Oct 3, 2020, at 04:41, Serhiy Storchaka wrote:
> Should we continue to support non-type objects with __bases__ in
> issubclass()? If no, can we remove their support in 3.10 (it would fix a
> crash in issue41909) or after passing a deprecation period?

Er, to be clear, contrary to the subject line, you're not suggesting removing 
support for non-type objects altogether [i.e. with __subclasscheck__], right? 
Just the ability to use a non-slotted __bases__ attribute?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BVDGIKBAM6U4CNGUP2BODSX62I7RTNLZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

[Random832]

On Tue, Sep 15, 2020, at 15:22, Mark Shannon wrote:
> 
> Hi all,
> 
> I'd like to propose a new PEP for hygienic macros.

To be clear, "hygienic macros" doesn't *just* mean AST-rewriting macros, it 
also means being able to define variables within the macro expansion that can't 
possibly collide with any names in the user code or in any other macros.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LHNATFDIXG5ZCWM7A2LEMDL6OQBG6WFA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Memory address vs serial number in reprs

[Random832]

On Sun, Jul 19, 2020, at 13:02, Guido van Rossum wrote:
> That looks expensive, esp. for objects implemented in Python — an extra 
> dict entry plus a new unique int object. What is the problem you are 
> trying to solve for these objects specifically? Just that the hex 
> numbers look distracting doesn’t strike me as sufficient motivation. 

Could the numbers be kept outside the object, perhaps in a weak* dictionary 
that's maintained in the __repr__ method, so you don't pay for it if you don't 
use it? *if the object's hash/eq use identity, anyway... a "weak identity-keyed 
dictionary" might be a nice thing to add anyway
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I34MYQDLIJNWEHV2JFO4QAKIZKCFTG7A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Recent PEP-8 change

[Random832]

On Thu, Jul 2, 2020, at 18:15, Antoine Pitrou wrote:
> On Thu, 02 Jul 2020 17:58:44 -0400
> Random832  wrote:
> > Why indeed?
> 
> Because we're talking about PEP 8, and PEP 8 intends to cover the code
> style used when writing code in the *Python standard library*.

ok, first of all, fair enough, but I do think there should be some recognition 
of the PEP's role in the wider community as a general style guide for all code 
written in python.

second, you've snipped the entire rest of my post rather than engage with it, 
and the main point I was making is no less relevant to the standard library, so 
I will repeat it: If the dialect someone wishes to write their comments in is 
mutually intelligible with "standard" (however defined) English there's no real 
need to enforce a higher degree of conformity beyond that.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FA4JMJTR7V745KGMTDK44GUBPE2HNP3V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Recent PEP-8 change

[Random832]

On Thu, Jul 2, 2020, at 05:20, Antoine Pitrou wrote:
> We're not talking about posting "your own writing", we're talking about
> comments (and presumably documentation) in a collective software
> project.  There's a need for consistency, however it's
> specified and achieved.
> 
> Otherwise why stop at English? I could just as well write my comments
> in French if it's all about individual freedom.  Requiring English is
> not inclusive, it forced people like me to painfully adapt to a
> language I wasn't used to.  And that has nothing to do with "white
> supremacy".

Why indeed?

Surely there are people somewhere in the world who write their comments in 
French, or Russian, or Japanese, and also name their variables in those 
languages, and I would argue there's nothing wrong with that (it certainly 
seems a lot of wasted effort supporting Unicode in variable names otherwise), 
they simply don't form a contiguous community with people whose code is in 
English

And that's the core, I think, of the issue. If the dialect someone wishes to 
write their comments in is mutually intelligible with "standard" (however 
defined) English there's no real need to enforce a higher degree of conformity 
beyond that. It can be understood, and that is enough. Whereas, if it is not, 
then they are effectively a foreign language programming community, and there's 
no reason to say they shouldn't go their own way any more than for 
French/Russian/Japanese/etc.

The desire to enforce a higher degree of conformity despite the lack of such a 
need is what has been described by some in this discussion as "white supremacy".
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZLFWE2NGCTUJ74FVAZLBIYG3AMTSK3VD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Emit a SyntaxWarning for unhashables literals in hashable dependant literals.

[Random832]

On Fri, Dec 13, 2019, at 02:20, mental na via Python-Dev wrote:
> Guido van Rossum wrote:
> > This is most definitely a language issue, not just a CPython issue -- the
> > rules around hashability and (im)mutability are due to the language
> > definition, not the whim of an implementer.
> 
> I was not aware of this, I assumed it was a implementation issue because 
> I knew CPython's dicts use a hash table implementation and there are other
> ways to implement a mapping data structure i.e. via trees.

There are significant differences in semantics between a hash table and a 
tree-based mapping. Dict has to be a hash table on all implementations. 
(However, Java mutable types are hashable and the sky hasn't fallen for them.)

> Guido could you provide a link to the language definition that dictates these
> rules about hashability and (im)mutability?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KESFBWNUT2ISHH6ETR65PLWPLXSEFYA2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP 611 - saturating reference counts

[Random832]

as an aside in the portion of PEP 611 proposing a limit to the number of 
classes, the proposal also mentioned reducing the size of the reference count 
field to achieve part of its object size savings.

one of the mechanisms proposed was to implement a "saturating" reference count, 
though it would be different from conventional saturating arithmetic in that 
any object that reaches the maximum refcount would become immortal, since we 
could not trust decref from there to account for all references.

something that may be worth considering or testing is: what would the cost, 
both in performance and interpreter code size, of implementing this logic on 
every single incref and decref?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7KUAXS7W245KM42Z6CUVTIAY5GNTENI5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Random832]

On Tue, Dec 3, 2019, at 12:22, Steve Dower wrote:
> > * The number of constants in a code object.
> 
> SGTM.

Two things...

First, for this one in particular, the number of constants in a code object is 
hard to predict. For example, recently (I want to say 3.7), the number of 
constants generated for most code was reduced by removing duplicate constants 
and intermediate constants that are optimized away by constant-folding. Adding 
the "x in [list or set literal]" optimization introduces a tuple or frozenset 
constant (and removes the constants for the items)

You also mentioned names - it's easy to imagine an implementation whose version 
of a code object shares names and constants in a single array with a single 
limit, or which has constants split up per type [and maybe doesn't implement 
tuple and frozenset constants at all] and shares names and string constants. Or 
one which does not store names at all for local variables in optimized code.

I also don't think characteristics of cpython-specific structures like code 
objects should be where limits are defined. One of the purposes of specifying a 
formal limit is to give other implementations a clear target for minimum 
support. In C, there's a long list of limits like these [some of which are 
quite a bit smaller than what typical implementations support], but all of them 
are for characteristics that can actually be determined by looking at the 
source code for a program.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/67OFA4G6A2X3XRBLG6XR4PWLEYBJLNLV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pass the Python thread state to internal C functions

[Random832]

On Mon, Nov 18, 2019, at 05:26, Antoine Pitrou wrote:
> > For the first goal, I don't think this is possible, or desirable.
> > Obviously if we remove the GIL somehow then at a minimum we'll need to
> > make the global threadstate a thread-local. But I think we'll always
> > have to keep it around as a thread-local, at least, because there are
> > situations where you simply cannot pass in the threadstate as an
> > argument. One example comes up when doing FFI: there are C libraries
> > that take callbacks, and will run them later in some arbitrary thread.
> > When wrapping these in Python, we need a way to bundle up a Python
> > function into a C function that can be called from any thread. So,
> > ctypes and cffi and cython all have ways to do this bundling, and they
> > all start with some delicate dance to figure out whether or not the
> > current thread holds the GIL, acquiring the GIL if not, then checking
> > whether or not this thread has a Python threadstate assigned, creating
> > it if not, etc. This is completely dependent on having the threadstate
> > available in ambient context. If threadstates were always passed as
> > arguments, then it would become impossible to wrap these C libraries.
> 
> Most well-designed C libraries let you pass an additional "void*"
> parameter for user callbacks to be called with.  A couple of them
> don't, unfortunately (OpenSSL perhaps?  I don't remember).

I think you've missed the fact that the C library runs the callback on an 
arbitrary thread. The threadstate associated with the thread that made the 
original call is therefore *not the one you want*; you want a threadstate 
associated with the thread the callback is run on.

Alternately, if a thread state is not in any sense associated with a thread 
(would these situations then mean you simply always create a brand-new 
interpreter state?), maybe it shouldn't be called a thread state at all.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NSRJVU7PZOVBWWYS3R5QLRRYP2N6NKTY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Pass the Python thread state to internal C functions

[Random832]

On Thu, Nov 14, 2019, at 07:43, Antoine Pitrou wrote:
> On Wed, 13 Nov 2019 14:52:32 +0100
> Victor Stinner  wrote:
> > 
> > #define _PyRuntimeState_GetThreadState(runtime) \
> > 
> > ((PyThreadState*)_Py_atomic_load_relaxed(&(runtime)->gilstate.tstate_current))
> > #define _PyThreadState_GET() _PyRuntimeState_GetThreadState(&_PyRuntime)
> > 
> > _PyThreadState_GET() uses "_Py_atomic_load_relaxed". I'm not used to
> > C99 atomic conventions. The "memory_order_relaxed" documentation says:
> > 
> > "Relaxed operation: there are no synchronization or ordering
> > constraints imposed on other reads or writes, only this operation's
> > atomicity is guaranteed (see Relaxed ordering below)"
> > 
> > Note: I'm not even sure why Python currently uses an atomic operation.
> 
> Is it protected by a lock?  If not, you need to use an atomic.
> Since it's theoretically possible to read the current thread state
> without the GIL held (though not very useful), then an atomic is
> required.

It sounds like you are saying PyRuntimeState_GetThreadState has two duties, 
then: "get this thread's thread state" (from the GIL holder - how do other 
threads get their own thread state), and "get the GIL-holding thread's thread 
state (from non-GIL holder thread).

The former shouldn't need atomic/overhead locking (unless the thread state can 
be written from other threads), even if the latter does.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4BNRSO47Z54MRR3ZS32W6DXYRVZ7U53W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migrate typing in builtin

[Random832]

On Mon, Oct 7, 2019, at 12:02, Philippe Prados wrote:
> Because this PEP propose to accept, for all classes
> assert isinstance("", int | str)
> assert issubclass(int, int | str)
> and add an operator __or__() for type type.
> def f(list: List[int | str], param: int | None) -> float | str:
> pass

Oh, sorry, I didn't realize that this also included the | operator, I thought 
this was just for isinstance("", Union[int, str]).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/52EL5VM6P3CZ7TMN7HOKUS7BCUE6HMFB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Migrate typing in builtin

[Random832]

On Mon, Oct 7, 2019, at 10:32, Philippe Prados wrote:
> Hello,
> 
> I try to implement a *proof of concept* of Pep-0604 
> .
> 
> To do this, the _GenericAlias must be in builtin, because I would like 
> to update issubclass() and isinstance() to accept Union.
> These methods are in builtin, and must be run without others modules.

Why can't this be handled as an __instancecheck__/__subclasscheck__?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UZRHLKTGR7USOD365N57Y2QVUWWMGPJ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: An f-string issue [Was: Re: Re: What to do about invalid escape sequences]

[Random832]

On Sat, Aug 10, 2019, at 19:54, Glenn Linderman wrote:
> Because of the "invalid escape sequence" and "raw string" discussion, 
> when looking at the documentation, I also noticed the following 
> description for f-strings:
> 
> > Escape sequences are decoded like in ordinary string literals (except when 
> > a literal is also marked as a raw string). After decoding, the grammar for 
> > the contents of the string is: followed by lots of stuff, followed by 
> > Backslashes are not allowed in format expressions and will raise an 
> error: f"newline: {ord('\n')}"  # raises SyntaxError 
>  What I don't understand is how, if f-strings are processed AS 
> DESCRIBED, how the \n is ever seen by the format expression.
> 
>  The description is that they are first decoded like ordinary strings, 
> and then parsed for the internal grammar containing {} expressions to 
> be expanded. If that were true, the \n in the above example would 
> already be a newline character, and the parsing of the format 
> expression would not see the backslash. And if it were true, that would 
> actually be far more useful for this situation.
> 
>  So given that it is not true, why not? And why go to the extra work of 
> prohibiting \ in the format expressions?

AIUI there were strong objections to the "AS DESCRIBED" process (which would 
require almost all valid uses of backslashes inside to be doubled, and would 
incidentally leave your example *still* a syntax error), and disallowing 
backslashes is a way to pretend that it doesn't work that way and leave open 
the possibility of changing how it works in the future without breaking 
compatibility.

The only dubious benefit to the described process with backslashes allowed 
would be that f-strings (or other strings, in the innermost level) could be 
infinitely nested as f'{f\'{f\\\'{...}\\\'}\'}', rather than being hard-limited 
to four levels as f'''{f"""{f'{"..."}'}"""}'''
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7NR4XYRTUAWFKGZEDTLODEWXV7YML6TZ/

[Python-Dev] Re: What to do about invalid escape sequences

[Random832]

On Mon, Aug 12, 2019, at 15:15, Terry Reedy wrote:
> Please no more combinations. The presence of both legal and illegal 
> combinations is already a mild nightmare for processing and testing. 
> idlelib.colorizer has the following re to detest legal combinations
> 
>  stringprefix = r"(?i:r|u|f|fr|rf|b|br|rb)?"

More advanced syntax highlighting editors have to handle each string type 
separately anyway, because they highlight (valid) backslash-escapes and 
f-string formatters. The proposed 'v-string' type would need separate handling 
even in a simplistic editor like IDLE, because it's different at the basic 
level of \" not ending the string (whereas, for better or worse, all current 
string types have exactly the same rules for how to find the end delimiter)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H77NNYCZI37JCHGSMIHMTKNQVK5SGCWY/

[Python-Dev] Re: What to do about invalid escape sequences

[Random832]

On Mon, Aug 5, 2019, at 00:29, raymond.hettin...@gmail.com wrote:
> The warning crops up frequently, often due to 
> third-party packages (such as docutils and bottle) that users can't 
> easily do anything about.  And during live demos and student workshops, 
> it is especially distracting.

Maybe what we really need is a general solution for warning visibility in 
third-party packages. Isn't that the whole reason DeprecationWarning isn't 
visible by default, for that matter?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GUULUXKYZN3UGWPEUPFKKEKNIZOOYJTJ/

[Python-Dev] Re: Comparing dict.values()

[Random832]

On Fri, Jul 26, 2019, at 05:45, Steven D'Aprano wrote:
> Nor should we make too much of the fact that Python sets require 
> elements to be hashable. As Java TreeSet demonstrates, we could get an 
> efficient set of unhashable items if we required orderability; and we 
> can get sets of unhashable, unorderable items if we're willing to 
> compromise on efficiency.

And think of what we could do if we were willing to compromise on immutability 
of hashable objects (Java does, in general, make the opposite decision there)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JJDKUFGILK2ESMUZULDB4I2VJFX3VQRC/

[Python-Dev] Re: Comparing dict.values()

[Random832]

On Fri, Jul 26, 2019, at 00:22, Ivan Pozdeev via Python-Dev wrote:
> Since a hash table is an unordered container and keys(), items() and 
> values() are iterators over it, *I would expect the results of any of 
> the comparisons to be undefined.*

keys, items, and values are not iterators. They are view objects, and keys and 
items implement the entire set API (in the same sense that range implements the 
entire sequence API). Values is the odd one out in that it can contain multiple 
instances of each object, so it can't really be considered as a set.

Items also sometimes contains unhashable types, and some methods simply fail in 
that case. I suggest that this precedent provides a way forward - implement the 
entire intuitive "contains the same amount of each value" algorithm [more or 
less Counter(obj1) == Counter(obj2)], and have this fail naturally, throwing 
e.g. an exception "TypeError: unhashable type: 'list'" if any of the values are 
unhashable in the same way that trying to perform certain set operations on an 
items view does.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UCW7SGSZPY2TNXYR2W7AB7NKAVTULZVT/

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

[Random832]

On Wed, May 29, 2019, at 01:25, Nick Coghlan wrote:
> Having a single locals() call de-optimize an entire function would be 
> far from ideal.

What if there were a way to explicitly de-optimize a function, rather than 
guessing the user's intent based on looking for locals and exec calls (both of 
which are builtins which could be shadowed or assigned to other variables)?

Also, regardless of anything else, maybe in an optimized function locals should 
return a read-only mapping?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: update 1

[Random832]

On Thu, May 23, 2019, at 15:27, Steve Holden wrote:
> Besides which, it would be lovely to have a major release that didn't 
> involve any pain at all for the majority of users!
> 
> Our erstwhile BDFL always eschewed two-digit version identifiers- due 
> to the possibility for confusion about collating sequence, I beleive.. 
> We should honour his preferences by going from 3.9 to 4.0.

Is 4.0 still gonna install as "python3", pip3, etc, executable as "py -3" on 
windows? I can see that being a pain point even if nothing else is.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Random832]

On Tue, May 21, 2019, at 14:17, Christian Heimes wrote:
> thanks for bringing this topic up. Initially I considered http.server, 
> too. But as Guido put it, it's both used and useful for local testing 
> and quick hacks. I'm already facing opposition for modules that are 
> less controversial and useful than http.server, too. 

Could the high-level functionality of http.server (and perhaps socketserver as 
a whole) be rebuilt on top of asyncio, with serve_forever implicitly starting 
an asyncio loop with the server as the only task? Or are the paradigms too 
different?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 572, Assignment Expressions

[Random832]

On Wed, Jul 11, 2018, at 20:10, Guido van Rossum wrote:
> As anticippated, after a final round of feedback I am hereby accepting PEP
> 572, Assignment Expressions: https://www.python.org/dev/peps/pep-0572/

I know everyone else is probably sick of discussing this (I somehow completely 
missed the discussion until it was almost done) but I have a question...

Why does this not allow assignment to attributes and subscripts? I think this 
is, at least, surprising enough that there should be a rationale section 
explaining it in the PEP. As it is, it's barely even explicitly stated, other 
than the use of 'NAME' in a few places and an offhand mention "and [the 
assignment statement] can assign to attributes and subscripts."
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exposing different versions of a system call in Python

[Random832]

On Fri, Jan 19, 2018, at 04:28, Pablo Galindo Salgado wrote:
> On the other side, we have pipe and pipe2 as an example of exposing two
> versions when this situation happens.
> 
> The question is:
> 
> What is preferable, exposing both functions or augment the old one?

A large number, possibly a majority, of system calls in the os module, have 
"dir_fd" arguments that cause them to call the *at counterpart of the 
underlying system call. And chdir can also be called with a file descriptor, 
which will call fchdir (though there is also a fchdir function) I don't know 
why pipe2 was implemented as a separate call.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?

[Random832]

On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote:
> Hi,
> 
> PEP 370 [1] was my first PEP that got accepted. I created it exactly one
> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have
> virtual environment support in Python. Ian Bicking had just started to
> create the virtualenv project a couple of months earlier.
> 
> Fast forward 10 years...
> 
> Nowadays Python has venv in the standard library. The user-specific
> site-packages directory is no longer that useful. I would even say it's
> causing more trouble than it's worth. For example it's common for system
> script to use "#!/usr/bin/python3" shebang without -s or -I option.
> 
> I propose to deprecate the feature and remove it in Python 4.0.

Where would pip install --user put packages, and how would one run scripts that 
require those packages? Right now these things Just Work; I've never had to 
learn how to use virtual environments.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Best Python API for exposing posix_spawn

[Random832]

On Mon, Jan 8, 2018, at 18:05, Gregory P. Smith wrote:
> i agree with just a list of tuples, but i suggest creating namedtuple
> instances in the posix module for the purpose (one each for close, dup2,
> open) .  Don't put a reference to a function in the tuple as Serhiy
> suggested as, while obvious what it means, it gives the wrong impression to
> the user: nothing is calling the Python functions.  This is a posix API
> that takes a list of arguments for a specific set of system calls for _it_
> to make for us in a specific order.

Instead of a sequence of functions to call, it'd be nice if a higher-level API 
could allow just passing in a mapping of file descriptor numbers to what they 
should point to in the new process, and the implementation figures out what 
sequence is necessary to get that result.

And at that point we could just extend the subprocess API to allow redirection 
of file descriptors other than 0/1/2, and have an implementation of it in terms 
of posix_spawn.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess not escaping "^" on Windows

[Random832]

On Sun, Jan 7, 2018, at 19:50, Steve Dower wrote:
> Considering there is no cross-platform compatibility here anyway, I 
> don’t think it’s that bad an option to let users do their own escaping, 
> especially since those who are successfully using this feature already 
> do.

I don't really think we should give up on cross-platform compatibility that 
easily. There are a number of constructs supported with the same syntax by both 
cmd and unix shells (pipes and redirection, mainly) that people may want to use.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Random832]

On Fri, Jan 5, 2018, at 14:07, Guido van Rossum wrote:
> Yeah, but I've changed my mind on this -- I think it's needless added
> complexity that helps save one line of code in very few use cases. And you
> don't really think the PEP endorses `nonlocal foo += 1` do you?

The PEP itself is very clear about supporting "augmented assignment", even 
defining the grammar differently for this case (such that you can't do it with 
multiple names in a single statement). The grammar shown also supports 
"nonlocal foo = bar = baz", though the text doesn't mention this form.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Whatever happened to 'nonlocal x = y'?

[Random832]

On Fri, Jan 5, 2018, at 10:47, Guido van Rossum wrote:
> I don't recall (though someone with more time might find the discussion in
> the archives or on the tracker). It was never implemented and I think it
> shouldn't be. So we might as well update the PEP. It wouldn't be
> particularly useful, since (by definition) the function that declares the
> nonlocal variable is not its owner, and hence it's unlikely to make sense
> to initialize it here. The same reasoning applies to global BTW.

I'm not so sure...

The only situation in which you're *required* to declare a nonlocal/global 
variable, after all, is if you intend to assign it - a name that you never 
assign is presumed to be non-local. The description in the PEP also applies to 
augmented assignments, and "global some_counter; some_counter += 1" is 
certainly a pattern I've needed in the past.

The PEP also quotes you as endorsing this for global. 
https://mail.python.org/pipermail/python-3000/2006-November/004166.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Usefulness of binary compatibility accross Python versions?

[Random832]

On Mon, Dec 18, 2017, at 05:23, Antoine Pitrou wrote:
> On Sun, 17 Dec 2017 21:07:02 -0500
> Random832 <random...@fastmail.com> wrote:
> > 
> > Is there any practical for of having the flag off for one slot and on
> > for another slot that's been added later?
> > 
> > Could this be replaced (that is, a slot for such a thing added before
> > it's too late) with a simple counter that goes up with each version, and
> > any "unused" slot should have NULL or some other sentinel value?
> 
> Any replacement here would break binary compatibility, which is what
> those flags are precisely meant to avoid.

I meant replacing the mechanism for new fields, rather than existing ones.

> > If it
> > really is important to have the flags themselves, just add another set
> > of flags - Py_TPFLAGS_HAVE_MORE_FLAGS.
> 
> Yes, we could... but it's more complication again.

Hmm, maybe that could be eased with macros...


/* Doing this preprocessor trick because if we used a ternary
 * operator, dummy macros needed to prevent compile errors may
 * become an attractive nuisance */
#define Py_TPFLAG_CHK(tp, flagname) \
Py__TPFCHK_##flagname(tp, flagname)
#define Py__TPFCHK_OLD(tp, flagname) \
((tp).tp_flags & Py_TPFLAGS_##flagname)
#define Py__TPFCHK_NEW(tp, flagname) \
((tp).tp_flags & Py_TPFLAGS_TPFLAGVER \
 && (tp).tp_flagver >= Py_TPFLAGVER_##flagname \
 && Py_TPFLAGCHK_##flagname(tp))

#define Py__TPFCHK_HEAPTYPE  Py__TPFCHK_OLD
#define Py_TPFLAGS_HEAPTYPE (1UL<<9)

#define Py__TPFCHK_TPFLAGVER Py__TPFCHK_OLD
#define Py_TPFLAGS_TPFLAGVER (1UL<<31)

#define Py__TPFCHK_NEWFLD Py__TPFCHK_NEW
#define Py_TPFLAGVER_NEWFLD 32
#define Py_TPFLAGCHK_NEWFLD(tp) ((tp).tp_newfld != NULL)

So to check heap type you get Py_TPFLAG_CHK(tp, HEAPTYPE)
((tp).tp_flags & (1UL<<9))

And to check newfld1 you get Py_TPFLAG_CHK(tp, NEWFLD)
((tp).tp_flags & (1UL<<31) && (tp).tp_flagver >= 32 && ((tp).tp_newfld != 
((void *)0)))

Or in a "more flags" scenario:

#define Py__TPFCHK_NEW(tp, flagname) \
((tp).tp_flags & Py_TPFLAGS_TPMOREFLAGS \
 && (tp).tp_moreflag & PY_TPMOREFLAGS_##flagname)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Usefulness of binary compatibility accross Python versions?

[Random832]

On Sat, Dec 16, 2017, at 08:22, Antoine Pitrou wrote:
> Typically, when adding a tp_XXX slot, you also need to add a
> Py_TPFLAGS_HAVE_XXX type flag to signal those static type structures
> that have been compiled against a recent enough PyTypeObject
> definition.  This way, extensions compiled against Python N-1 are
> supposed to "still work": as they don't have Py_TPFLAGS_HAVE_XXX set,
> the core Python runtime won't try to access the (non-existing) tp_XXX
> member.

Is there any practical for of having the flag off for one slot and on
for another slot that's been added later?

Could this be replaced (that is, a slot for such a thing added before
it's too late) with a simple counter that goes up with each version, and
any "unused" slot should have NULL or some other sentinel value? If it
really is important to have the flags themselves, just add another set
of flags - Py_TPFLAGS_HAVE_MORE_FLAGS.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Random832]

On Fri, Dec 1, 2017, at 05:31, Steven D'Aprano wrote:
> I'm more confused than ever. You seem to be arguing that Python 
> functions CAN short-circuit their arguments and avoid evaluating them. 
> Is that the case?

> If this is merely about when the name "function" is looked up, then I 
> don't see why that's relevant to the PEP.
> 
> What am I missing?

You're completely missing the context of the discussion, which was the
supposed reason that a *new* function call operator, with the proposed
syntax function?(args), that would short-circuit (based on the
'function' being None) could not be implemented. The whole thing doesn't
make sense to me anyway, since a new operator could have its own
sequence different from the existing one if necessary.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Random832]

On Thu, Nov 30, 2017, at 17:26, Greg Ewing wrote:
> Eric Fahlgren wrote:
> > ​I think you're missing something here, since it seems clear to me that 
> > indeed the arguments are evaluated prior to the function call.​
> 
> I think the OP may be confusing "evaluating the function" with
> "calling the function".
> 
> If the function being called is determined by some computation,
> that computation may be performed before its arguments are
> evaluated (and is probably required to be, by the "left to
> right" rule). But the arguments will always be evaluated
> before the actual call happens.

Right, but if the function is evaluated before either of those things
happen, then it can indeed short-circuit the argument evaluation.

The OP isn't confusing anything; it's Eric who is confused. The quoted
paragraph of the PEP clearly and unambiguously claims that the sequence
is "arguments -> function -> call", meaning that something happens after
the "function" stage [i.e. a None check] cannot short-circuit the
"arguments" stage. But in fact the sequence is "function -> arguments ->
call".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Random832]

On Thu, Nov 30, 2017, at 11:08, Eric Fahlgren wrote:
> On Thu, Nov 30, 2017 at 2:48 AM, Andrea Griffini  wrote:
> 
> > Not really related but the PEP says that arguments in Python are evaluated
> > before the function (as a reason to reject the idea of None-aware function
> > call) but this is not the case:
> >
> 
> ​I think you're missing something here, since it seems clear to me that
> indeed the arguments are evaluated prior to the function call.​  Maybe
> unrolling it would help?  This is equivalent to the body of your lambda,
> and you can see that the argument is evaluated prior to the call which
> receives it.

Of course they're evaluated prior to the function *call*, but the pep
says they're evaluated prior to the function *itself* [i.e. arg = g();
func = f(); func(arg)].
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Random832]

On Tue, Nov 28, 2017, at 15:31, Raymond Hettinger wrote:
> 
> > I also cc python-dev to see if anybody here is strongly in favor or against 
> > this inclusion.
> 
> Put me down for a strong -1.   The proposal would occasionally save a few
> keystokes but comes at the expense of giving Python a more Perlish look
> and a more arcane feel.   
> 
> One of the things I like about Python is that I can walk non-programmers
> through the code and explain what it does.  The examples in PEP 505 look
> like a step in the wrong direction.  They don't "look like Python" and
> make me feel like I have to decrypt the code to figure-out what it does.
> 
> timeout ?? local_timeout ?? global_timeout
> 'foo' in (None ?? ['foo', 'bar'])
> requested_quantity ?? default_quantity * price
> name?.strip()[4:].upper()
> user?.first_name.upper()

Since we're looking at different syntax for the ?? operator, I have a
suggestion for the ?. operator - and related ?[] and ?() that appeared
in some of the proposals. How about this approach?

Something like (or None: ...) as a syntax block in which any operation
[lexically within the expression, not within e.g. called functions, so
it's different from simply catching AttributeError etc, even if that
could be limited to only catching when the operand is None] on None that
is not valid for None will yield None instead.

This isn't *entirely* equivalent, but offers finer control.

v = name?.strip()[4:].upper() under the old proposal would be more or
less equivalent to:

v = name.strip()[4:].upper() if name is not None else None

Whereas, you could get the same result with:
(or None: name.strip()[4:].upper())

Though that would technically be equivalent to these steps:
v = name.strip if name is not None else None
v = v() if v "
v = v[4:] """
v = v.upper """
v = v() """

The compiler could optimize this case since it knows none of the
operations are valid on None. This has the advantage of being explicit
about what scope the modified rules apply to, rather than simply
implicitly being "to the end of the chain of dot/bracket/call operators"

It could also be extended to apply, without any additional syntax, to
binary operators (result is None if either operand is None) (or None: a
+ b), for example, could return None if either a or b is none.

[I think I proposed this before with the syntax ?(...), the (or None:
...) is just an idea to make it look more like Python.]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default

[Random832]

On Tue, Nov 7, 2017, at 07:22, Nick Coghlan wrote:
> My suggestion for that definition is to have the *default* meaning of
> "third party code" be "everything that isn't __main__".

What is __main__? Or, rather, how do you determine when it is to blame?
For syntax it's easy, but any deprecated function necessarily belongs to
its own module and not to main. Main may have called it, which can be
detected from the stack trace, or it may have used it in some other way
(pass to some builtin or e.g. itertools function that takes a callable
argument, for example). Maybe the DeprecationWarning should be raised at
the name lookup* rather than the call? What if "calling this function
with some particular combination of arguments" is deprecated?


*i.e. something like:

class deprecated:
   def __init__(self, obj): self.obj = obj
class DeprecatableModule(ModuleType):
   def __getattr__(self, name):
  obj = self.__dict__[name]
  if isinstance(type(obj), deprecated):
  if (detect somehow caller is __main__): raise
  DeprecationWarning
  return obj.obj
  else: return obj
def __dir__(self):
return [k for k in self.__dict__ if not
isinstance(self.__dict__[k], deprecated)]
  
sys.modules[__name__].type=DeprecatableModule

@deprecated
def some_deprecated_function(...): ...

SOME_DEPRECATED_CONSTANT = deprecated(42)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The current dict is not an "OrderedDict"

[Random832]

On Tue, Nov 7, 2017, at 09:56, Steven D'Aprano wrote:
> Don't let the perfect be the enemy of the good.
> 
> For many applications, keys are never removed from the dict, so this 
> doesn't matter. If you never delete a key, then the remaining keys will 
> never be reordered.
> 
> I think that Nick's intent was not to say that after a single deletion, 
> the ordering guarantee goes away "forever", but that a deletion may be 
> permitted to reorder the keys, after which further additions will honour 
> insertion order. At least, that's how I interpret him.

Honestly, I think the more intuitive way would be the "other way around"
- deletions don't themselves change the order, but they do cause
subsequent insertions to be allowed to insert at places other than the
end.

Does the implementation in CPython have this property?

One way I have seen this done is that the items themselves live in an
array, and each new item is always inserted in the first empty slot in
the array (empty slots form a freelist). The hash buckets contain
indices into the array.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] iso8601 parsing

[Random832]

On Wed, Oct 25, 2017, at 16:32, Alexander Belopolsky wrote:
> This is annoying when you deal with time
> series where it is common to have text files with a mix of dates,
> timestamps and numbers.  You can write generic code to deal with ints
> and floats, but have to special-case anything time related.

Generic code that uses a Callable[[str], ...] instead of a type works
fine with a class method.

column1.parser = int
column2.parser = float
column3.parser = datetime.parse_iso
column4.parser = json.loads

It is *very slightly* more complex than a model that needs the type also
for some reason and has the type pull double duty as the parser... but
why do that?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What is the design purpose of metaclasses vs code generating decorators? (was Re: PEP 557: Data Classes)

[Random832]

On Fri, Oct 13, 2017, at 02:30, Nick Coghlan wrote:
> Metaclasses currently tend to serve two distinct purposes:
>
> 1. Actually altering the runtime behaviour of a class and its children
>in non-standard ways (e.g. enums, ABCs, ORMs)
> 2. Boilerplate reduction in class definitions, reducing the amount of
>code you need to write as the author of that class
>
> Nobody has a problem with using metaclasses for the first purpose -
> that's what they're for.
>
> It's the second use case where they're problematic, as the fact that
> they're preserved on the class becomes a leaky implementation detail,
> and the lack of a JIT in CPython means they can also end up being
> expensive from a runtime performance perspective.

What about a metaclass that isn't a metaclass? A metaclass can be any
callable and can return any object, e.g. a normal type.

def AutoSlotMeta(name, bases, dct, real_metaclass=type):
"""turn all class variables into slots"""
dct['__slots__'] = list(dct)
return real_metaclass(name, bases, dct)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Need help to fix urllib(.parse) vulnerabilities

[Random832]

On Fri, Jul 21, 2017, at 08:43, Giampaolo Rodola' wrote:
> It took me a while to understand the security implications of this
> FTP-related bug, but I believe I got the gist of it here (I can
> elaborate further if it's not clear):
> https://github.com/python/cpython/pull/1214#issuecomment-298393169
> My proposal is to fix ftplib.py and guard against malicious
> strings involving the *PORT command only*. This way we fix the
> issue *and* maintain backward compatibility by allowing users to
> specify "\n" in their paths and username / password pairs. Java
> took a different approach and disallowed "\n" completely. To my
> understanding fixing ftplib would automatically mean fixing urllib
> as well. 

What would a \n in a path mean? What commands would you send over FTP to
successfully retrieve a file (or enter a username or password)
containing a newline in the name? In other words, what exactly are we
being backward compatible *with*?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Format strings, Unicode, and Py2.7: need clarification

[Random832]

On Thu, May 18, 2017, at 01:14, Hobson Lane wrote:
> Because `.format()` is a method on an instantiated `str` object in e and
> so
> must return the same type

That it *must* return the same type is overstating the matter. Split
returns a list (and, rather like %, the list is of unicode or str
objects depending on the argument). Join will return a unicode object if
any of the elements of the sequence are unicode. I was honestly
surprised though to see that % returns unicode when formatting a unicode
value, since my mental model of %s was more like {!s} - call str() on
whatever object is at the given position in the right-hand argument.
This kind of ad hoc implementation decision (format always returns str,
other methods can return unicode,  ljust/rjust refuse to accept a
unicode character argument) is what Python 3 moved away from.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols

[Random832]

On Mon, Mar 20, 2017, at 14:07, Brett Cannon wrote:
> What is a "trivial body"? I don't know of any such definition anywhere in
> Python so this is too loosely defined. You also don't say what happens if
> the body isn't trivial. Are tools expected to raise an error?

My assumption would be that a trivial body is any body consisting of
only a docstring and/or a "pass" statement.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538: Coercing the legacy C locale to a UTF-8 based locale

[Random832]

On Tue, Mar 14, 2017, at 10:17, Nick Coghlan wrote:
> It's not that you *can't* run Python 3 in that kind of environment, and
> it's not that there are never any valid reasons to do so. It's that lots
> of
> things that you'd typically expect to work are going to misbehave (one I
> discovered myself yesterday is that the GNU readline problems reported in
> interactive mode on Android also show up when you do either "LANG=C
> python2" or "LANG=C python3" on traditional Linux and attempt to *edit*
> lines containing multi-byte characters)

It occurs to me that (at least for readline... and maybe also as a
general proxy for whether the rest should be done) detecting the IUTF8
terminal flag (which, properly, controls basic non-readline-based line
editing such as backspace) may be worthwhile.

(And maybe Readline itself should be doing this, more or less
independent of Python. But that's a discussion for elsewhere)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Impoverished compare ...

[Random832]

On Thu, Mar 9, 2017, at 18:43, Erik wrote:
> Hi.
> 
> I'm looking at stuff proposed over on Python-Ideas, and I'd appreciate 
> some pointers as to the basics of how C-level objects are generally 
> compared in Python 3.
> 
> The issue is related to the performance of PyObject_RichCompare. I got 
> to the point where I was trying to work out what was the _alternative_ 
> to RichCompare. If a comparison is not "rich", then what is it? There's 
> a tp_richcompare slot in the type structure, but I haven't noticed 
> anything else obvious for simple comparison (In 2.x days - which I have 
> more experience with - __cmp__ was a thing which now seems to be gone. I 
> understand the Python-level changes with sort(..., key=foo) but I've not 
> looked at the underlying C implementation until now).

In 2.x the C version of __cmp__ was tp_compare (and it existed even in
python 0.9.1, which had neither dunder methods nor tp_richcompare).

I assume that the "rich compare" name was retained in Python 3 for the
same reason that other names like PyLongObject, PyUnicodeObject were
(not PyStringObject, though, presumably because they don't want people
to unintentionally create a bytes-only API as a result of a lazy porting
process). Maybe Python 4000 can alias it to tp_compare (PyIntObject,
PyStringObject) and Python 5000 can get rid of the current names.

It's "rich" because it knows which of the object-level methods (less
than, greater than, etc) is being called, whereas tp_compare/__cmp__ did
not.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] List mutation in list_repr?

[Random832]

On Tue, Dec 6, 2016, at 05:27, Patrick Westerhoff wrote:
> Hey all,
> 
> I just stumbled on the following comment in the C source of the repr
> implementation for the list object:
> 
> /* Do repr() on each element. Note that this may mutate the list,
>so must refetch the list size on each iteration. */
> 
> (as seen in list_repr implementation [1])
> 
> I’m honestly very surprised about this remark since I can neither
> understand why this would be the case (repr shouldn’t mutate the
> list)

It *shouldn't*, but it can't be enforced. It's one of those things where
if Python assumes all user code is sane (in this case, overridden
__repr__ not messing with the list) it can bite in a way that could
cause the interpreter to crash.

>>> class EvilClass:
...  def __repr__(x):
...   l.pop()
...   return 'x'
...
>>> l = [EvilClass()]*10
>>> l
[x, x, x, x, x]

>, and I also don’t see any clue in the source as to when this
> would actually happen. Since inside the loop, the list object `v` is
> never accessed other than passing `v->ob_item[i]` to the recursive
> repr call, there shouldn’t be any mutation on the list object itself.

The item may have or be able to a reference to the list object
otherwise, as I demonstrated.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

[Random832]

On Mon, Oct 10, 2016, at 14:04, MRAB wrote:
> Instead of locking the object, could we keep the GIL, but have it 
> normally released?
> 
> A thread could then still call a function such as PyWeakref_GetObject() 
> that returns a borrowed reference, but only if it's holding the GIL. It 
> would be able to INCREF the reference before releasing the GIL again.

So, what stops the other thread which never asks for the GIL from
blowing away the reference? Or is this a special kind of lock that you
can "assert isn't locked" without locking it for yourself, and
INCREF/DECREF does so?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TextIO seek and tell cookies

[Random832]

On Mon, Sep 26, 2016, at 05:30, Ben Leslie wrote:
> I think the case of JSON or SQL database is even more important though.
> 
> tell/seek can return 129-bit integers (maybe even more? my maths might
> be off here).
> 
> The very large integers that can be returned by tell() will break
> serialization to JSON, and storing in a SQL database (at least for
> most database types).
> 
> What is the value of comparing these to plain integers? Unless you
> happen to know the magic encoding it isn't going to be very useful I
> think?

I assume the value is that in the circumstances in which all of the
flags and other bits are zero, they can be used as offsets in precisely
the way that you used them. It may also be possible that in some cases
where they are not zero, doing arithmetic with them is still "safe"
since the real offset is still in the low-order bits. I don't know if
those circumstances are predictable enough for it to be worthwhile.
Changing it would obviously break code that does this (unless, perhaps,
it were changed to be a class with arithmetic operators), the question
is whether such code "deserves" to be broken.

In my own tests, even a UTF-8-sig file with DOS line endings "worked".
Does anyone have information about what circumstances can reliably cause
tell() to return values that are *not* simple integers? Maybe it has
something to do with working with stateful encodings like iso-2022 or
UTF-7?

What was the situation that caused your problem?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Drastically improving list.sort() for lists of strings/ints

[Random832]

On Tue, Sep 13, 2016, at 13:24, Nikolaus Rath wrote:
> On Sep 11 2016, Terry Reedy  wrote:
> > Tim Peters investigated and empirically determined that an
> > O(n*n) binary insort, as he optimized it on real machines, is faster
> > than O(n*logn) sorting for up to around 64 items.
> 
> Out of curiosity: is this test repeated periodically on different
> architectures? Or could it be that it only ever was true 10 years ago on
> Tim's Power Mac G5 (or whatever he used)?

Binary insertion sort is still O(n*logn) in comparisons, so it's likely
that this is due to short memmoves being sufficiently fast due to cache
effects as not to matter. The number might have gotten larger or
smaller, though. I wonder if it's something that could be tuned
dynamically, at compile time or install time.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (some) C99 added to PEP 7

[Random832]

On Thu, Sep 8, 2016, at 16:01, Chris Barker wrote:
> Is there a "long" in there anywhere in the integer implementation?

The python 2 long type is the python 3 int type. The python 2 int type
is gone.

> I don't have py3 running on win64 anywhere right now, but in win64 py2,
> that would give you:
> 
> dtype('int32')
> 
> as it's a "long" under the hood

That's numpy's decision, there's nothing "built-in" about it.

> (and I'm pretty sure that is not because of numpy code itself, but rather
> how Cpython is written/compiled)

Nope.
https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/common.c#L105

Note that PyInt_Check doesn't exist anymore in Python 3. NumPy provides
its own definition:

https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/npy_3kcompat.h#L35
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 529: Change Windows filesystem encoding to UTF-8

[Random832]

On Thu, Sep 8, 2016, at 13:10, Guido van Rossum wrote:
> On Thu, Sep 8, 2016 at 9:57 AM, Brett Cannon  wrote:
> > Bash on Windows is just Linux, so it isn't affected by any of this.
> 
> I don't know what that sentence means.

It means that the so-called "bash" on windows 10 is actually a full
Ubuntu system (running on, AIUI, a simulation of Linux kernel system
calls), which will presumably also have its own python installation and
use a UTF-8 locale, rather than one that runs "natively" on win32.

If it's possible for a win32 version of python to call it as a
subprocess, this may be an argument in favor of using UTF-8 - subject to
finding out whether WSL does use UTF-8, whether it supports non-ASCII
arguments from a Win32 CreateProcess at all, whether there's any way to
pass non-UTF-8 arguments to it, etc.

Incidentally, according to
https://github.com/Microsoft/BashOnWindows/issues/2, pipes didn't work
at all between WSL processes and Win32 processes until two weeks ago, so
it's clear that these features are still evolving.

> But anyways, if someone wants
> to try making subprocess work with bytes arguments on Windows work,
> that's just a bugfix, and you're not constrained by how it works on
> previous Python versions (since it doesn't work there at all). It
> might be wise to choose an interpretation that's consistent with other
> uses of command line arguments by Python on Windows though (rather
> than choosing to favor making just bash work the same as it works on
> Linux).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (some) C99 added to PEP 7

[Random832]

On Thu, Sep 8, 2016, at 12:30, Chris Barker wrote:
> That's why I said "based on" -- under the hood, a C type is used, and
> IIUC, that type has been "long" for ages. And a long on Windows 64
> (with the MS compiler anyway) is 32 bit, and a long on *nix (with the
> gnu compilers, at least) is 64 bits.
>
> This doesn't expose itself to pure python (and sys.maxint is now gone)
> but it does get exposed in the C API, and in particular, when passing
> data back and forth between numpy and pure python (numpy doesn't
> support an unlimited integer like python), or working with buffers or
> bytearrays, or whatever in Cython.

I'm not sure "the builtin integer type" was the right term for what
you're referring to.

You're talking about changing Py_ssize_t, right?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

[Random832]

On Tue, Sep 6, 2016, at 06:34, Martin Panter wrote:
> Yes, that was basically it. Though I had only thought as far as simple
> encodings like ASCII, where one byte corresponds to one character. I
> wonder if you really need UTF-8 support. Are the encoding values
> currently encountered for Windows consoles all single-byte encodings
> or are they more complicated?

Windows supports Chinese, Japanese, and Korean encodings (code pages
936, 932, 949) that are multi-byte with one or two bytes per character.
I'm not sure how that affects this though or what your point about not
needing UTF-8 is.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Random832]

On Sun, Sep 4, 2016, at 16:42, Koos Zevenhoven wrote:
> On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
> >
> > There are two self-consistent sets of names:
> >
> 
> Let me add a few. I wonder if this is really used so much that
> bytes.chr is too long to type (and you can do bchr = bytes.chr if you
> want to):
> 
> bytes.chr (or bchr in builtins)
> bytes.chr_at, bytearray.chr_at

Ugh, that "at" is too reminiscent of java. And it just feels wrong to
spell it "chr" rather than "char" when there's a vowel elsewhere in the
name.

Hmm... how offensive to the zen of python would it be to have "magic" to
allow both bytes.chr(65) and b'ABCDE'.chr[0]? (and possibly also
iter(b'ABCDE'.chr)? That is, a descriptor which is callable on the
class, but returns a view on instances?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Random832]

On Sat, Sep 3, 2016, at 18:06, Koos Zevenhoven wrote:
> I guess one reason I don't like bchr (nor chrb, really) is that they
> look just like a random sequence of letters in builtins, but not
> recognizable the way asdf would be.
> 
> I guess I have one last pair of suggestions for the name of this
> function: bytes.chr or bytes.char.

What about byte? Like, not bytes.byte, just builtins.byte.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Random832]

On Sat, Sep 3, 2016, at 08:08, Martin Panter wrote:
> On 1 September 2016 at 19:36, Ethan Furman  wrote:
> > Deprecation of current "zero-initialised sequence" behaviour without removal
> > 
> >
> > Currently, the ``bytes`` and ``bytearray`` constructors accept an integer
> > argument and interpret it as meaning to create a zero-initialised sequence
> > of the given size::
> >
> > >>> bytes(3)
> > b'\x00\x00\x00'
> > >>> bytearray(3)
> > bytearray(b'\x00\x00\x00')
> >
> > This PEP proposes to deprecate that behaviour in Python 3.6, but to leave
> > it in place for at least as long as Python 2.7 is supported, possibly
> > indefinitely.
> 
> Can you clarify what “deprecate” means? Just add a note in the
> documentation, or make calls trigger a DeprecationWarning as well?
> Having bytearray(n) trigger a DeprecationWarning would be a minor
> annoyance for code being compatible with Python 2 and 3, since
> bytearray(n) is supported in Python 2.

I don't think bytearray(n) should be deprecated. I don't think that
deprecating bytes(n) should entail also deprecating bytes(n).

If I were designing these classes from scratch, I would not feel any
impulse to make their constructors take the same arguments or have the
same semantics, and I'm a bit unclear on what the reason for this
decision was.

I also don't think bytes.fromcount(n) is necessary. What's wrong with
b'\0'*n? I could swear this has been answered before, but I don't recall
what the answer was. I don't think the rationale mentioned in the PEP is
an adequate explanation, it references an earlier decision, about a
conceptually different class (it's an operation that's much more common
with mutable classes than immutable ones - when's the last time you did
(None,)*n relative to [None]*n), without actually explaining the real
reason for either underlying decision (having bytearray(n) and having
both classes take the same constructor arguments).

I think that the functions we should add/keep are:
bytes(values: Union[bytes, bytearray, Iterable[int]) 
bytearray(count : int)
bytearray(values: Union[bytes, bytearray, Iterable[int]) 
bchr(integer)

If, incidentally, we're going to add a .fromsize method, it'd be nice to
add a way to provide a fill value other than 0. Also, maybe we should
also add it for list and tuple (with the default value None)?

For the (string, encoding) signatures, there's no good reason to keep
them [TOOWTDI is str.encode] but no good reason to get rid of them
either.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Random832]

On Fri, Sep 2, 2016, at 19:44, Ethan Furman wrote:
> The problem with only having `bchr` is that it doesn't help with
> `bytearray`;

What is the use case for bytearray.fromord? Even in the rare case
someone needs it, why not bytearray(bchr(...))?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What should a good type checker do? (was: Please reject or postpone PEP 526)

[Random832]

On Fri, Sep 2, 2016, at 18:49, Koos Zevenhoven wrote:
> Then again, (b) instead of that being working code, it might be an
> error and spam only takes float. No problem, the type checker will
> catch that.

There are very few functions that should only take float and not int.

> On Fri, Sep 2, 2016 at 9:04 PM, Steven D'Aprano 
> wrote:
> > Maybe you think that it's okay because ints and floats are somewhat
> > compatible. But suppose I wrote:
> >
> > if cond:
> > x = HTTPServer(*args)
> > else:
> > x = 1.5
> 
> It might be clear by now, but no, that's not why I wrote that. That
> was just a slightly more "realistic" example than this HTTP & 1.5 one.

The other thing is... I'd kind of want it to infer Number in the first
case.

And if I assign both a list and a generator expression to something,
that should be Iterable[whatever] and maybe even whatever gets worked
out for "proper iterable and not string or bytes or memoryview".

Certainly if you can return HTTPServer() or None it should infer
Union[HTTPServer, None], otherwise spelled Optional[HTTPServer].

Maybe what we need is a "protoize"-alike that can run through a source
file and produce a stub file with all its inferences, for manual
inspection. So if you see something nonsensical like Union[HTTPServer,
float] you can think "wait a minute, where's that coming from" and go
look at the code.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

[Random832]

On Thu, Sep 1, 2016, at 18:28, Steve Dower wrote:
> This is a raw (bytes) IO class that requires text to be passed encoded
> with utf-8, which will be decoded to utf-16-le and passed to the Windows APIs.
> Similarly, bytes read from the class will be provided by the operating 
> system as utf-16-le and converted into utf-8 when returned to Python.

What happens if a character is broken across a buffer boundary? e.g. if
someone tries to read or write one byte at a time (you can't do a
partial read of zero bytes, there's no way to distinguish that from an
EOF.)

Is there going to be a higher-level text I/O class that bypasses the
UTF-8 encoding step when the underlying bytes stream is a console? What
if we did that but left the encoding as mbcs? I.e. the console is text
stream that can magically handle characters that aren't representable in
its encoding. Note that if anything does os.read/write to the console's
file descriptors, they're gonna get MBCS and there's nothing we can do
about it.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supported versions of OpenSSL

[Random832]

On Mon, Aug 29, 2016, at 04:09, M.-A. Lemburg wrote:
> Hmm, that last part would mean that Python 3.7 will no longer compile
> on e.g. Ubuntu 14.04 LTS which uses OpenSSL 1.0.1 as default version.
> Since 14.04 LTS is supported until 2019, I think it would be better
> to only start requiring 1.0.2 in Python 3.8.

If someone's going to compile a new version of Python for Ubuntu LTS,
they can compile a new version of OpenSSL. How likely is it that someone
will want to use the packaged version of OpenSSL, but not use the
packaged version (which is 3.4) of Python?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Failures in test_site.py - how to debug?

[Random832]

On Fri, Aug 19, 2016, at 18:10, Chris Angelico wrote:
> On Sat, Aug 20, 2016 at 8:07 AM, Sjoerd Job Postmus 
> wrote:
> > I'd like to re-iterate my suggestion in case it was missed: split the
> > current test in 2 tests:
> >
> > * Running with `-S` which is for checking that by default the collections
> > are not imported. (Which is what is currently tested)
> > * Running without `-S` which is for checking that `site` gets imported.
> >
> > I hope that is clear enough. Do you see any problems with such an approach?
> 
> AIUI this wouldn't test that site is sufficiently slim. The test is
> supposed to mimic normal startup, then assert that unnecessary modules
> haven't been loaded. Splitting it would prove that a -S run is nice
> and slim, but would allow bloat in site.py to pass unnoticed.

What about -S and putting "import site" explicitly in the test code? Or
would that go back to importing everything on people who have packages
installed?

Really, there should be a "source-isolated" mode, which removes
/usr/lib/whatever from the path in the same way that -I removes the
user's stuff, but still runs cpython/Lib/site.py , and all tests (and
for that matter stuff like frozen module prep) should be run in this
mode.

I'm not sure I see a downside to having such a mode be the *default*
mode for "python detects it is being run from a source tree".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Failures in test_site.py - how to debug?

[Random832]

On Fri, Aug 19, 2016, at 10:13, Chris Angelico wrote:
> On my main dev system (Debian Stretch), I've had a single
> long-standing test failure - test_site.py,
> StartupImportTests.test_startup_imports. It's annoying (partly because
> it's such a noisy failure), and doesn't appear to be happening on the
> buildbots, nor presumably on core devs' computers, so it's probably
> not a Python bug. How can I go about figuring out what's actually
> going on here?
> 
> The test effectively runs this (but with -v, which makes it noisy):

the -v output might be helpful in determining what is causing these
modules to be imported. It would at least show what order they're
imported in.

FWIW, the list of modules that yours has that are not present in my
3.5.2:
- _bootlocale _collections _functools _heapq _locale _operator
- collections collections.abc functools heapq itertools keyword
- mpl_toolkits operator reprlib types weakref

Modules that mine has imported but yours does not:
- _sysconfigdata sitecustomize

> It then runs into a failure when it asserts that this set has no
> intersection with the set of collection_mods. Apparently 'import sys'
> on my system (even with -I for isolation mode) loads up a bunch of
> modules that it shouldn't be loading.

I very much doubt that "import sys" is responsible here.

> How do I track down what and
> why, and figure out whether it's a config problem unique to my system
> or not?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] stuck issue 26826

[Random832]

On Wed, Aug 3, 2016, at 16:32, Marcos Dione wrote:
> (it needs to check the availability of the function and the suitability
> for the parameters given; copy_file_range() only works on files on the
> same filesystem[1]). Hmm...

What is the benefit to using copy_file_range over sendfile in this
scenario? Or does sendfile not work with regular files on Linux?

Maybe os.sendfile should use copy_file_range if available/applicable,
and the shutils functions can use it?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Convert from unsigned long long to PyLong

[Random832]

On Fri, Jul 22, 2016, at 11:50, Eric Snow wrote:
> On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring  wrote:
> > So to sum this up, you claim that PyLong_FromUnsignedLongLong can
> > somehow produce a number larger than the value range of a 64 bit
> > number (0x10180). I have a hard time believing this.
> 
> Perhaps I misunderstood your meaning, but Python's integers (AKA
> "PyLong") can be bigger that a machine-native integer (e.g. 64 bits):

Yes, but you can't (or shouldn't be able to) get those values from a
conversion from a machine-native type such as
PyLong_FromUnsignedLongLong.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should we fix these errors?

[Random832]

On Fri, Jul 22, 2016, at 11:35, Victor Stinner wrote:
> Oh, the first one is a regression that I introduced in the
> implementation of the PEP 475 (retry syscall on EINTR). I don't think
> that it can be triggered in practice, because socket handles on
> Windows are small numbers, so unlikely to be seen as negative.

The problem as I understand it isn't that handles will be seen as
negative, the problem is that the error return will be seen as
*non*-negative.

> I just fixed it:
> https://hg.python.org/cpython/rev/6c11f52ab9db

Does INVALID_SOCKET exist on non-windows systems? (It's probably safe to
compare against -1, the relevant functions are defined in POSIX as
returning -1 rather than an unspecified negative value)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: next round

[Random832]

On Mon, Jul 18, 2016, at 17:34, Alexander Belopolsky wrote:
> On Mon, Jul 18, 2016 at 5:01 PM, Jonathan Goble 
> wrote:
> 
> > full(), despite its use in numpy, is also unintuitive to me (my first
> > thought is that it would indicate whether an object has room for more
> > entries).
> >
> > Perhaps bytes.fillsize?
> 
> I wouldn't want to see bytes.full() either.  Maybe bytes.of_size()?

What's wrong with b'\0'*42?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] When to use EOFError?

[Random832]

On Mon, Jun 27, 2016, at 12:40, Ethan Furman wrote:
> On 06/21/2016 01:48 PM, Serhiy Storchaka wrote:
> 
> > There is a design question. If you read file in some format or with some
> > protocol, and the data is ended unexpectedly, when to use general
> > EOFError exception and when to use format/protocol specific exception?
> 
> I believe that EOFError was created for the situation when a file 
> unexpectedly ends.

The problem is that's not a good abstraction for the class of errors
we're discussing, because it means you've got to pick: the thing your
parser parses is a file [and non-files are supported by wrapping them in
a StringIO/BytesIO] or it is a str/bytes [and files are supported by
reading their data into a string].

Or you could use a third option: a method that accepts a file raises
EOFError, and a method that accepts a string raises some other error
(ValueError?), and if either is implemented in terms of the other it's
got to wrap the exception.

(Also, that's nonsense. EOFError is also used when a file *expectedly*
ends - EAFP i.e. "Exceptions As Flow-control is Pythonic" ;)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

[Random832]

On Fri, Jun 24, 2016, at 17:52, Eric Snow wrote:
> - 2 open questions (__slots__?  drop read-only requirement?)

It's worth noting that __slots__ itself doesn't have a read-only
requirement. It can be a tuple, any iterable of strings, or a single
string (which means the object has a single slot).

Should dir() iterate in the order of __definition_order__? What, if so,
should be done about instance attributes, or attributes of multiple
classes, or class attributes not present in __definition_order__?

What happens to classes whose __prepare__ doesn't return an OrderedDict?

Can __definition_order__ be reassigned at runtime? Will it have the same
constraints?

What if a metaclass defines __getattribute__ in a way that specially
handles __definition_order__? If someone really wants to put a non-tuple
there they will find a way. How hard do we want to think about ways to
stop consenting adults from doing weird things with the
__definition_order__ attribute?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why are class dictionaries not accessible?

[Random832]

On Wed, Jun 22, 2016, at 11:11, Guido van Rossum wrote:
> This is done in order to force all mutations of the class dict to go
> through attribute assignments on the class. The latter takes care of
> updating the class struct, e.g. if you were to add an `__add__` method
> dynamically it would update tp_as_number->nb_add. If you could modify the
> dict object directly it would be more difficult to arrange for this side
> effect.

Why is this different from the fact that updating a normal object's dict
bypasses descriptors and any special logic in __setattr__? Dunder
methods are already "special" in the sense that you can't use them as
object attributes; I wouldn't be surprised by "assigning a dunder method
via the class's dict breaks things".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] When to use EOFError?

[Random832]

On Tue, Jun 21, 2016, at 16:48, Serhiy Storchaka wrote:
> There is a design question. If you read file in some format or with some 
> protocol, and the data is ended unexpectedly, when to use general 
> EOFError exception and when to use format/protocol specific exception?
> 
> For example when load truncated pickle data, an unpickler can raise 
> EOFError, UnpicklingError, ValueError or AttributeError. It is possible 
> to avoid ValueError or AttributeError, but what exception should be 
> raised instead, EOFError or UnpicklingError? Maybe convert all EOFError 
> to UnpicklingError?

I think this is the most appropriate. If the calling code needs to know
the original reason it can find it in __cause__.

My instinct, though, (and I'm aware that others may not agree, but I
thought it was worth bringing up) is that loads should actually always
raise a ValueError, i.e. my mental model of loads is like:

def loads(s):
   f = BytesIO(s)
   try:
  return load(f)
   except UnpicklingError as e:
  raise ValueError from e
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Why are class dictionaries not accessible?

[Random832]

The documentation states: """Objects such as modules and instances have
an updateable __dict__ attribute; however, other objects may have write
restrictions on their __dict__ attributes (for example, classes use a
dictproxy to prevent direct dictionary updates)."""

However, it's not clear from that *why* direct dictionary updates are
undesirable. This not only prevents you from getting a reference to the
real class dict (which is the apparent goal), but is also the
fundamental reason why you can't use a metaclass to put, say, an
OrderedDict in its place - because the type constructor has to copy the
dict that was used in class preparation into a new dict rather than
using the one that was actually returned by __prepare__.

[Also, the name of the type used for this is mappingproxy, not
dictproxy]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Our responsibilities (was Re: BDFL ruling request: should we block forever waiting for high-quality random bits?)

[Random832]

On Thu, Jun 16, 2016, at 07:34, Donald Stufft wrote:
>   python-dev tends to favor not breaking “working” code over securing
>   existing  APIs, even if “working” is silently doing the wrong thing
>   in a  security  context. This is particularly frustrating when it
>   comes to security  because  security is by it’s nature the act of
>   taking code that would  otherwise  execute and making it error,
>   ideally only in bad situations, but this  “security’s purpose is to
>   make things break” nature clashes with  python-dev’s  default of
>   not breaking “working” code in a way that is personally  draining
>   to me.

I was almost about to reply with "Maybe what we need is a new zen of
python", then I checked. It turns out we already have "Errors should
never pass silently" which fits *perfectly* in this situation. So what's
needed is a change to the attitude that if an error passes silently,
that making it no longer pass silently is a backward compatibility
break.

This isn't Java, where the exceptions not thrown by an API are part of
that API's contract. We're free to throw new exceptions in a new version
of Python.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Random832]

On Thu, Jun 16, 2016, at 10:04, Barry Warsaw wrote:
> On Jun 16, 2016, at 09:51 AM, Random832 wrote:
> 
> >On Thu, Jun 16, 2016, at 04:03, Barry Warsaw wrote:
> >> *If* we can guarantee that os.urandom() will never block or raise an
> >> exception when only poor entropy is available, then it may be indeed
> >> indistinguishably backward compatible for most if not all cases.  
> >
> >Why can't we exclude cases when only poor entropy is available from
> >"most if not all cases"?
>
> Because if it blocks or raises a new exception on poor entropy it's an
> API break.

Yes, but in only very rare cases. Which as I *just said* makes it
backwards compatible for "most" cases.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Random832]

On Thu, Jun 16, 2016, at 04:03, Barry Warsaw wrote:
> *If* we can guarantee that os.urandom() will never block or raise an
> exception when only poor entropy is available, then it may be indeed
> indistinguishably backward compatible for most if not all cases.

Why can't we exclude cases when only poor entropy is available from
"most if not all cases"?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does base64 return bytes?

[Random832]

On Tue, Jun 14, 2016, at 22:58, Stephen J. Turnbull wrote:
> The RFC is unclear on this point, but I read it as specifying the
> ASCII coded character set, not the ASCII repertoire of (abstract)
> characters.  Therefore, it specifies an invertible mapping from a
> particular set of integers to characters.

There are multiple descriptions of base 64 that specifically mention
using it with EBCDIC and with local character sets of unspecified
nature.

>  > The intention is clearly to represent binary data as *text*.
> 
> It's more subtle than that.  *RFCs do not deal with text.*  Text is
> an internal concept of (some) programming environments.

It's also a human concept. Plenty of RFCs deal with human concept rather
than purely programming topics.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does base64 return bytes?

[Random832]

On Tue, Jun 14, 2016, at 13:19, Paul Sokolovsky wrote:
> Well, it's easy to remember the conclusion - it was decided to return
> bytes. The reason also wouldn't be hard to imagine - regardless of the
> fact that base64 uses ASCII codes for digits and letters, it's still
> essentially a binary data. 

Only in the sense that all text is binary data. There's nothing in the
definition of base64 specifying ASCII codes. It specifies *characters*
that all happen to be in ASCII's character repertoire.

>And the most natural step for it is to send
> it down the socket (socket.send() accepts bytes), etc.

How is that more natural than to send it to a text buffer that is
ultimately encoded (maybe not even in an ASCII-compatible encoding...
though probably) and sent down a socket or written to a file by a layer
that is outside your control? Yes, everything eventually ends up as
bytes. That doesn't mean that we should obsessively convert things to
bytes as early as possible.

I mean if we were gonna do that why bother even having a unicode string
type at all?

> I'd find it a bit more surprising that binascii.hexlify() returns
> bytes, but I personally got used to it, and consider it a
> consistency thing on binascii module.
> 
> Generally, with Python3 by default using (inefficient) Unicode for
> strings, 

Why is it inefficient?

> any efficient data processing would use bytes, and then one
> appreciates the fact that data encoding/decoding routines also return
> bytes, avoiding implicit expensive conversion to strings.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does base64 return bytes?

[Random832]

On Tue, Jun 14, 2016, at 13:05, Joao S. O. Bueno wrote:
> Sorry, it is 2016, and I don't think at this point anyone can consider
> an ASCII string
> as a representative pattern of textual data in any field of application.
> Bytes are not text. Bytes with an associated, meaningful, encoding are
> text.
>   I thought this had been through when Python 3 was out.

Of all the things that anyone has said in this thread, this makes the
*least* contextual sense. The input to base64 encoding, which is what is
under discussion, is not text in any way. It is images, it is zip files,
it is executables, it could be the output of os.urandom (at least,
provided it doesn't block ;) for all anyone cares.

The *output* is only an ascii string in the sense that it is a text
string consisting of characters within (a carefully chosen subset of)
ASCII's repertoire, but the output wasn't what he was claiming should be
bytes in the sentence you replied to. Is your objection to the phrase
"ascii string"?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Random832]

On Sat, Jun 11, 2016, at 22:37, Theodore Ts'o wrote:
> On Fri, Jun 10, 2016 at 05:14:50PM -0400, Random832 wrote:
> > So, I have a question. If this "weakness" in /dev/urandom is so
> > unimportant to 99% of situations... why isn't there a flag that can be
> > passed to getrandom() to allow the same behavior?
> 
> The intention behind getrandom() is that it is intended *only* for
> cryptographic purposes. 

I'm somewhat confused now because if that's the case it seems to
accomplish multiple unrelated things. Why was this implemented as a
system call rather than a device (or an ioctl on the existing ones)? If
there's a benefit in not going through the non-atomic (and possibly
resource limited) procedure of acquiring a file descriptor, reading from
it, and closing it, why is that benefit not also extended to
non-cryptographic users of urandom via allowing the system call to be
used in that way?

> Anyway, if you don't need cryptographic guarantees, you don't need
> getrandom(2) or getentropy(2); something like this will do just fine:

Then what's /dev/urandom *for*, anyway?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Stop using timeit, use perf.timeit!

[Random832]

On Fri, Jun 10, 2016, at 21:45, Steven D'Aprano wrote:
> If you express your performances as speeds (as "calculations per 
> second") then the harmonic mean is the right way to average them.

That's true in so far as you get the same result as if you were to take
the arithmetic mean of the times and then converted from that to
calculations per second. Is there any other particular basis for
considering it "right"?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

[Random832]

On Fri, Jun 10, 2016, at 15:54, Theodore Ts'o wrote:
> So even on Python pre-3.5.0, realistically speaking, the "weakness" of
> os.random would only be an issue (a) if it is run within the first few
> seconds of boot, and (b) os.random is used to directly generate a
> long-term cryptographic secret.  If you are fork openssl or ssh-keygen
> to generate a public/private keypair, then you aren't using os.random.

So, I have a question. If this "weakness" in /dev/urandom is so
unimportant to 99% of situations... why isn't there a flag that can be
passed to getrandom() to allow the same behavior?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] C99

[Random832]

On Mon, Jun 6, 2016, at 07:23, Sturla Molden wrote:
> Ok, but if third-party developers shall be free to use a C89 compiler for
> their own code, we cannot have C99 in the include files. Otherwise the
> include files will taint the C89 purity of their source code.
> 
> Personally I don't think we need to worry about compilers that don't
> implement C99 features like inline functions in C. How long have the
> Linux
> kernel used inline functions instead of macros? 20 years or more?

Using inline functions instead of macros doesn't have to mean anything
but a performance hit on platforms that don't support them, since the
inline keyword, or some other identifier, could be defined to expand to
an empty token sequence on platforms that do not support it. It's much
lower impact on the source code than some other C99 features.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does PEP 7/8 explicitly suggest 2 spaces after a period?

[Random832]

On Sun, May 22, 2016, at 11:15, Peter Ludemann via Python-Dev wrote:
> I just tried a re-flow in emacs (M-q or fill-paragraph) and it didn't
> change my single space to two (it also changed 3 spaces after a period
> to 2; and it preserved my hanging indents).

It won't change a single space to two, but it will add two spaces after
a sentence which was previously at the end of a line... and it *won't*
break a line after a dot followed by a single space (Vim isn't clever
enough for this second bit), preferring to break before the preceding
word instead.

A dot followed by a single space is *permitted* (hence why you observed
it didn't change it to two), it is simply assumed to represent something
other than the end of a sentence, such as an initial in a name.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] File system path PEP, part 2

[Random832]

On Fri, May 13, 2016, at 07:00, Steven D'Aprano wrote:
> - but os.fspath() will only return str;
> 
> - and os.fspathb() will only return bytes;

And raise an exception if __fspath__ returns the other, I suppose.
What's the use case for these functions? When would I call them rather
than fsdecode and fsencode? (assuming the latter will support the path
protocol - I've got more objections if they're not going to)

Also, what happens if you pass a string to os.fspath? Statements like
"str will not implement __fspath__" have thus far been light on details
of what functions will accept "str or
path-like-object-whose-__fspath__returns-str" and which ones will only
accept the latter (and what, if any, will continue to only accept the
former). If no-one's supposed to directly call dunder methods, and
os.fspath accepts str, then what difference does it make whether this is
implemented by having a special case within os.fspath or by calling
str.__fspath__?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Convert int() to size_t in Python/C

[Random832]

On Fri, Apr 29, 2016, at 14:11, Marcos Dione wrote:
> These are not output parameters, even if they're pointers. they'r
> using the NULL pointer to signal that the current offsets should not be
> touched, to differentiate from a offset of 0. Something that in Python we
> would use None.

That's not actually true according to the documentation. (And if it
were, they could simply use -1 rather than a null pointer)

If you pass a null pointer in, the file's offset is used and *is*
updated, same as if you used an ordinary read/write call. If you pass a
value in, that value is used *and updated* (which makes it an output
parameter) and the file's offset is left alone.

Documentation below, I've >>>highlighted<<< the part that shows they are
used as output parameters:

   The following semantics apply for off_in, and similar statements
   apply to off_out:

   *  If off_in is NULL, then bytes are read from fd_in starting
   from
  the file offset, and the file offset is adjusted by the number
  of
  bytes copied.

   *  If off_in is not NULL, then off_in must point to a buffer that
  specifies the starting offset where bytes from fd_in will be
  read.
  The file offset of fd_in is not changed, >>>but off_in is
  adjusted
  appropriately.<<<
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Convert int() to size_t in Python/C

[Random832]

On Fri, Apr 29, 2016, at 10:45, Marcos Dione wrote:
> One possible solution hat was suggested to me in the #python IRC
> channel was to use that, then test if the resulting value is negative,
> and adjust accordingly, but I wonder if there is a cleaner, more general
> solution (for instance, what if the type was something else, like loff_t,
> although for that one in particular there *is* a convertion
> function/macro).

In principle, you could just use PyLong_AsUnsignedLong (or LongLong),
and raise OverflowError manually if the value happens to be out of
size_t's range. (99% sure that on every linux platform unsigned long is
the same size as size_t.

But it's not like it'd be the first function in OS to call a system call
that takes a size_t. Read just uses Py_ssize_t. Write uses the buffer
protocol, which uses Py_ssize_t. How concerned are you really about the
lost range here? What does the system call return (its return type is
ssize_t) if it writes more than SSIZE_MAX bytes? (This shouldn't be hard
to test, just try copying a >2GB file on a 32-bit system)

I'm more curious about what your calling convention is going to be for
off_in and off_out. I can't think of any other interfaces that have
optional output parameters. Python functions generally deal with output
parameters in the underlying C function (there are a few examples in
math) by returning a tuple.

Maybe return a tuple (returned value, off_in, off_out), where None
corresponds to the input parameter having been NULL (and passing None in
makes it use NULL)?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inconsistency of PyModule_AddObject()

[Random832]

On Thu, Apr 28, 2016, at 10:11, Stefan Krah wrote:
> For actual users of Valgrind this is patently obvious and was
> pretty much the point of my post.

A more relevant point would be that _decimal does *not* use the API in a
way *which would be broken by the proposed change*, regardless of
whether the way in which it uses it is subjectively correct or can cause
leaks.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inconsistency of PyModule_AddObject()

[Random832]

On Thu, Apr 28, 2016, at 05:05, Stefan Krah wrote:
> $ valgrind --suppressions=Misc/valgrind-python.supp ./python -c "import
> decimal"
> 
> [...]
> ==16945== LEAK SUMMARY:
> ==16945==definitely lost: 0 bytes in 0 blocks
>  

Well, the obvious flaw with your test case is that a reference is
retained forever in the C static variable basic_context_template.

Now, it is arguable that this may be a reasonably common pattern, and
that this doesn't actually constitute misuse of the API (the reference
count will be wrong, but the object itself is immortal anyway, so it
doesn't matter if it's 2 or 1 since it can't be 0 even with correct
usage)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()

[Random832]

On Mon, Apr 18, 2016, at 15:26, Stephen J. Turnbull wrote:
> in
> particular it is not true for DirEntry (which is a "enhanced
> degenerate" path containing only one path segment but also other
> useful information abot the filesystem object addressed)

DirEntry contains multiple path segments - it has the name, and the
directory path that was passed into scandir.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 8 updated on whether to break before or after a binary update

[Random832]

On Fri, Apr 15, 2016, at 23:46, Peter Ludemann via Python-Dev wrote:
> If Python ever adopts the BCPL rule for implicit line continuation if
> the last thing on a line is an operator (or if there's an open
> parentheses), then the break-after-an-operator rule would be more
> persuasive. ;)
>
> [IIRC, the BCPL rule was that there was an implicit continuation if
> the grammar would not allow inserting a semicolon at the end of the
> line, which covered both the open-parens and last-item-is-operator
> cases, and probably a few others.] But I should shut up and leave shut
> discussions to python-ideas.

Sounds like Visual Basic. Meanwhile, Javascript's rule is that there's
an implicit semicolon if and only if the grammar would not allow the
two lines to be considered as a single statement. Insanity comes in
all flavors.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] RFC: PEP 509: Add a private version to dict

[Random832]

On Fri, Apr 15, 2016, at 16:41, Victor Stinner wrote:
> If the dictionary values are modified during the loop, the dict
> version is increased. But it's allowed to modify values when you
> iterate on *keys*.

Why is iterating over items different from iterating over keys?

in other words, why do I have to write:

for k in dict:
v = dict[k]
...do some stuff...
dict[k] = something

rather than

for k, v in dict.items():
...do some stuff...
dict[k] = something


It's not clear why the latter is something you want to prevent.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pathlib - current status of discussions

[Random832]

On Thu, Apr 14, 2016, at 13:56, Koos Zevenhoven wrote:
> (1) Code that has access to pathname/filename data and has some level
> of control over what data type comes in. This code may for instance
> choose to deal with either bytes or str
> 
> (2) Code that takes the path or file name that it happens to get and
> does something with it. This type of code can be divided into
> subgroups as follows:
> 
>   (2a) Code that accepts only one type of paths (e.g. str, bytes or
> pathlib) and fails if it gets something else.

Ideally, these should go away.

>   (2b) Code that wants to support different types of paths such as
> str, bytes or pathlib objects. This includes os.path.*, os.scandir,
> and various other standard library code. Presumably there is also
> third-party code that does the same. These functions may want to
> preserve the str-ness or bytes-ness of the paths in case they return
> paths, as the stdlib now does. But new code may even want to return
> pathlib objects when they get such objects as inputs.

Hold on. None of the discussion I've seen has included any way to
specify how to construct a new object representing a different path
other than the ones passed in. Surely you're not suggesting type(a)(b).

Also, how does DirEntry fit in with any of this?

> This is the
> duck-typing or polymorphic code we have been talking about. Code of
> this type (2b) may want to avoid implicit conversions because it makes
> the life of code of the other types more difficult.

As long as the type it returns is still a path/bytes/str (and therefore
can be accepted when the caller passes it somewhere else) what's the
problem?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] MAKE_FUNCTION simplification

[Random832]

On Thu, Apr 14, 2016, at 12:56, Terry Reedy wrote:
> https://docs.python.org/3/library/dis.html#module-dis
> CPython implementation detail: Bytecode is an implementation detail of 
> the CPython interpreter. No guarantees are made that bytecode will not 
> be added, removed, or changed between versions of Python.
> 
> Version = minor release, as opposed to maintenance release.

"between versions" is ambiguous. It could mean that there's no guarantee
that there will be no changes from one version to the next, or it could
mean, even more strongly, that there's no guarantee that there will be
no changes in a maintenance release (which are, after all, released
*between* minor releases)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pathlib - current status of discussions

[Random832]

On Thu, Apr 14, 2016, at 12:05, Stephen J. Turnbull wrote:
> Random832 writes:
> 
>  > And what such incompatibilities exist between bytes and str for the
>  > purpose of representing file paths?
> 
> A plethora of encodings.

Only one encoding, fsencode/fsdecode. All other encodings are not for
filenames.

>  > At the end of the day, there's exactly one answer to "what file on
>  > disk this represents (or would represent if it existed)".
> 
> Nope.  Suppose those bytes were read from a file or a socket?  It's
> dangerous to assume that encoding matches the file system's.

Why can I pass them to os.open, then, or to os.path.join so long as
everything else is also bytes?

On UNIX, the filesystem is in bytes, so saying that bytes can't match
the filesystem is absurd. Converting it to str with fsdecode will
*always, absolutely, 100% of the time* give a str that will address the
same file that the bytes does (even if it's "dangerous" to assume that
was the name the user wanted, that's beyond the scope of what the module
is capable of dealing with).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pathlib - current status of discussions

[Random832]

On Thu, Apr 14, 2016, at 09:50, Chris Angelico wrote:
> Adding integers and floats is considered "safe" because most people's
> use of floats completely compasses their use of ints. (You'll get
> OverflowError if it can't be represented.) But float and Decimal are
> considered "unsafe":
> 
> >>> 1.5 + decimal.Decimal("1.5")
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unsupported operand type(s) for +: 'float' and
> 'decimal.Decimal'
> 
> This is more what's happening here. Floats and Decimals can represent
> similar sorts of things, but with enough incompatibilities that you
> can't simply merge them.

And what such incompatibilities exist between bytes and str for the
purpose of representing file paths? At the end of the day, there's
exactly one answer to "what file on disk this represents (or would
represent if it existed)".
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com