[Python-ideas] Deprecate PEP 249 (DB-API 2.0)

[Stephen J. Turnbull]

Hi, Soni

Interesting idea.  Sure does appear to be low-hanging fruit.

But I'm not sure anybody who matters is still listening here.  Either
a merge request or posting on the relevant Discourse channel is more
likely to attract interest.

Steve

Soni L. writes:
 > We would like to propose the following improvements to DB-API 2.0 that 
 > would require bumping it up to DB-API 3.0:
 > 
 > - Get rid of SQL strings
 > - Get rid of SQL strings
 > - Use package resources to store what would otherwise be SQL strings
 > 
 > While we cannot prevent someone from going out of their way to define 
 > package resources at runtime just so they can implement SQL injection, 
 > ultimately the goal is to provide a small speed bump so they don't feel 
 > so inclined to jump straight into SQL injection before trying to do 
 > easier, more secure things.
 > ___
 > Python-ideas mailing list -- python-ideas@python.org
 > To unsubscribe send an email to python-ideas-le...@python.org
 > https://mail.python.org/mailman3/lists/python-ideas.python.org/
 > Message archived at 
 > https://mail.python.org/archives/list/python-ideas@python.org/message/STPNELT3ZP337ELTDTL7QR43N7BZOWXV/
 > Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5RQ6TO2OBTYFDDUHISEDBRAGVUV2IIT6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: re.match(pattern, string, require=True)

[Stephen J. Turnbull]

Chris Angelico writes:

 > Why create a new argument, then mandate that you use it everywhere,
 > just to achieve what's already happening?

"Newbies don't read code backwards very well" seems to be the
point.

While I'm not of the school that "I learned this painfully, so newbies
should learn this painfully", I do think that novice Python
programmers should learn that

1.  "None has no .xxx attribute" means that some previous code (often
but not always a regex match) was unable to perform some task
and returned None to indicate failure.
2.  If the failure was expectable, your code is buggy because it
didn't test for None, and if it was unexpected, some code
somewhere is buggy because it allowed an invariant to fail.

On the cost side, there are so many cases where a more finely divided
Exception hierarchy would help novices quite a bit but experts very
little that this case (easy to learn) would open the floodgates.  I
believe Guido has specifically advised against such a hierarchy.  I'm
against this change.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MIE4OFPAG5CTNMUR7FYJSX66UMDHIH57/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: SyntaxError: cannot use assignment expressions with attribute

[Stephen J. Turnbull]

Dom Grigonis writes:

 > Why would this not be a good option? 1 extra line compared to
 > walrus, but no DRY issue.
 >   with open(“fn") as f:
 > while True:
 > line = f.readline()
 > if line and check(line):
 > process(line)
 > else:
 > break

Works for me.  I suspect you can construct situations where it would
be a lot uglier.  But I'm not a fan of the walrus in the first place,
so I'm not going to take this any further.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ATQJORWTV5PJ2Z43FIPP5E2Z4GZWQGAS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: SyntaxError: cannot use assignment expressions with attribute

[Stephen J. Turnbull]

Dom Grigonis writes:

 > Mistake, late night. I just meant to portray initialisation via
 > walrus inside statement, nothing more. It doesn’t work with while
 > loop.

It *does* work with a while loop, just not that one.  See below.  The
problem is that you're writing new code full of trash you don't need
to explain the concept.  Keep these examples as simple as possible.
For example, my "pass" example is too simple if you want to show how
the walrus can save typing because it doesn't save any lines of code;
you need something a little more complicated to demonstrate that
saving.

In the progress of any proposal to acceptance, at some point someone
is going to say, "I see that it works and solves a theoretical
problem, but who would ever use it?"  At that point you go into a
large body of code such as the Python standard library or numpy to
find realistic examples.  But that's old code, and only needed to
demonstrate actual utility.

 > Generally, this would be useful for all examples of
 > https://peps.python.org/pep-0572/ 
 > , where “initial” value is an attribute or dictionary item.

That's not the question though.  The question is are those examples
themselves frequently useful.  And that's when you go find them in the
stdlib.

 > I see. Could you give an example of this? I vaguely remember this
 > sometimes being the case, but can not find anything now on this in
 > relation to walrus operator. Is there no way to bring that
 > execution within the body without walrus operator?

The problem is code like this:

with open("a_file") as f:
while (line := f.readline())
if check(line):
process(line)
else:
break

There is no way to do that without an assignment, and without the
walrus you need an assignment before the while, and the same
assignment within the loop body.  Is that a big deal?  No.  In fact, I
have never used the walrus operator in anger, nor wanted to.  I'm just
as happy to write

with open("a_file") as f:
line = f.readline()
while (line)
if check(line):
process(line)
else:
break
line = f.readline()

But some people find loop-and-a-half extremely ugly, and while I
differ on "extremely", I can't deny that it's ugly.

You *could* do this for small files with

with open("a_file") as f:
lines = f.readlines()
for (line in lines):
if check(line):
process(line)
else:
break

but that's not possible with an infinite iterable, undesireable for
most non-file streams, etc.

Re "simple examples", see why I used the "if check()" stuff?  If I
wasn't going to talk about infinite iterables and pausing external
streams, that would just be (potentially wrong!) complexity that
doesn't help explain anything.

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3SSXFFCSGLMJ4HBDJXOP6KFJDCGEXZDE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: SyntaxError: cannot use assignment expressions with attribute

[Stephen J. Turnbull]

Dom Grigonis writes:

 > This nuance can also be encountered in “principal use-case”. E.g.:
 > class A:
 > def func(self):
 > while (self.a := 1) < 5:
 > self.a += 1
 > return self.a

Not sure what you're getting at here, that's an infloop.  Did you mean
something like this:

class A:
def func(self):
while (self.a := self.a += 1) < 5:
pass
return self.a

I'm pretty sure that exact idiom, modeled on the C "copy string" idiom

void strcpy(char *s, char *t) {
while (*t++ = *s++)

would generally be considered unpythonic, but I suppose some people
might like it if instead of pass you had a real suite there.  I think
it's a real "meh" use case, since in most cases it can be rewritten

class A:
def func(self):
while self.a < (5 - 1):  # expression to make the
 # transformation clear
self.a += 1
return self.a

and you save a couple characters and at most one line (none in the
case you present).

 > Same argument as for “walrus” operator itself - convenient
 > feature. Or is there more to it?

There's more to it.  The loop and a half construct, where you execute
the body of the loop once outside the loop for some reason is widely
considered a really big wart (DRY failure).  The walrus allows us to
eliminate most of those.

On the other hand, an assignment and a return statement are two
different things; it's not a DRY viotion to have the same identifier
in both.

 > I was actually surprised that this didn’t work - I thought that
 > this operator is a literal composite of assignment and “retriever",
 > rather than having it’s own restrictive logic.

This is a common practice in Python development.  Try a little bit of
a new idea, avoid engineering, and expand later if that seems useful
too.

 > If it doesn’t break anything, doesn’t have any undesirable side
 > effects and community likes it, then could be a good addition.

"Community likes it" is a null predicate, very hard to verify in edge
cases.  When the support in the community is strong enough that
"community likes it" is common knowledge, it's always the case that
there are objective reasons why it's a good thing.

All additions have a 0 * infinity cost: a negligible cost of learning
(for one user) times *all* the users.

Other projects feel differently about it, but Python tends to be quite
conservative about additions.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IFTS6XVVNAYXBMUBUB2FF6QX7BMFKXVQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Reconstructing datetime from microsecond timestamp

[Stephen J. Turnbull]

Samuel Freilich via Python-ideas writes:

 > This might all be too much thought about edge cases that don't
 > matter, but given the *_ns() functions in the time module (PEP
 > 564), I'm curious why datetime doesn't have a constructor that
 > takes an integer timestamp with the full precision that supports.

This is such an obvious improvement that the only answer I can think
of is "Because you haven't submitted a merge request yet?" ;-)

TBH honest you're right about the edge case, mostly datetime is about
talking to humans and that's almost by definition not worth the time
of a core dev to shave nanoseconds off the operation.  But if someone
is looking for a chance to put in a patch, this looks likely to be
approved to me.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EDJQVPIIPR2CWOANXMNTPKODHQHCEBOW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Dom Grigonis writes:

 > > But it's far from concise
 > What could be more concise?

A notation where you don't have to repeat a possibly long expression.
For example, numerical positions like regular expressions.  Consider
this possible notation:

f'There are {count} expression{pluralize(count)} denoted by {=0}.'

Otherwise it isn't great, but it's definitely concise.  In the
simplest case you could omit the position:

f'{=} is {count} at this point in the program.'

 > > and violates DRY -- it doesn't solve the problem of the first
 > > draft typo.

 > And how is “postfix =“ different?

You *can't* use different identifiers for the name and value in
"postfix =": the same text is used twice, once as a string and one as
an identifier.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HNARI3L4PXNTJFGZD5AHTBG2RBLIC5OD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Dom Grigonis writes:

 > By “elegant", I wasn’t talking about the syntax.

Neither was I, except in the comment about "mnemonic".  I use "postfix
=" and "prefix =" because I don't know of better names that indicate
the semantics of the feature.

Semantically, "prefix =" is a reasonable solution to the problem --
assuming you consider it a problem.  But it's far from concise and
violates DRY -- it doesn't solve the problem of the first draft typo.
I don't see it as elegant the way "postfix =" is.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NB66RQQID7HKB5NOG6BRGQMZCPDHBQ7P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Dom Grigonis writes:

 > Eric Smith wrote:
 >> Since I wrote that commit: no one is saying it’s impossible or
 >> overly difficult,

 > To be honest it is exactly what was being said.

Sure ... about an unclearly expressed early version of the proposal,
that seemed to ask "given an object x, tell me the name of x".  In the
end, I don't think there was any disagreement that doing that reliably
is indeed impossible.  But if someone has written that the #nameof
version of the proposal is impossible to implement, that is not
representative of what most of us think or have said.

 > What I think is that it would be an elegant feature if it was
 > implemented at python level.

The postfix '=' flag is elegant by anyone's standard, I think: it
provides immediately useful, though limited, functionality, with no
waste, and is automatically fit for purpose because it satisfies DRY.
And it is mnemonic in the sense that you may have to look it up to
write it, but once you've learned it, you will recognize it when you
see it in unfamiliar code because it "looks like" what it produces.

The proposed prefix '=' flag is much less attractive to me on all
counts above, except that it's quite mnemonic.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7MVQ27WLEDWEDKKTSWM4VBWMOENXSIOJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Dom Grigonis writes:

 > Simply eval/exec string inputs. I sometimes use of these for ad-hoc
 > callbacks. Would be great if `v` was recognised as a code.

Looking at the example, do you mean '=' here?  That is, you want to
add semantics for prefixed '=', meaning "just insert the expression
string here"?

 > Code is similar to this: class A:
 > def __init__(self):
 > self.d = {'a': 1, 'b': 2}
 > 
 > def apply(self, smtp):
 > for k, v in self.d.items():
 > if callable(smtp):
 > self.d[k] = smtp(v)
 > elif isinstance(smtp, str):
 > self.d[k] = eval(f'{=v}{smtp}')

Since f'{=v}' just results in 'v', why isn't

self.d[k] = eval(f'v{smtp}')

fine?  Sure, you get the compiler check/refactoring benefit, but as
you admit, this is a very "ad hack" use case.  I don't think we should
encourage it in production code.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PEVTGJOPRWP5SLBLK4UZHISD3EGTRR7K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Dom Grigonis writes:

 > print(f'{=A.a}')# 'A.a'
 > print(nameof(A.a))  # 'a'

I see that's the C# semantics, but it's not obvious to me why the
ambiguity introduced is worth emulating.  The important aspect of the
proposed 'nameof' operator is that its argument can be validated by
the compiler and unambiguously recognized by refactoring tools.  But
once it's a string, it loses those advantages.  Why not be unambiguous?

I don't think that the tricks recommended (ie, typeof) to retrieve a
fully-qualified name in C# would be unambiguous in Python.

Also, Python's f-string '=' operator handles a broad variety of
expressions, not just attribute references.  This has been useful to
me more often than bare names and attribute references.

 > Both [member name and full reference] seem useful to me.

What am I missing?  When would you want the member name only but
"nameof().split('.')[-1]" would not do?

Also, are there use cases for the proposed nameof other than a more
flexible f-string '='?  Ie, where only the str is wanted, not the
str-ified object?  That clearly doesn't apply to the refactoring
application, though, unless you're communicating local names to
nonlocal computations or using eval, both of which seem like really
dubious practices to me.

 > So the functionality is bound to an editor and anyone using another
 > editor would just see a weird string?

No, they'd see an ordinary string.  Editors would use heuristics to
recognize strings like f"the value of expression is {expression}".  I
do stuff like this in Emacs all the time ad hoc, Chris is just
suggesting that some editors/IDEs would provide a more powerful and
accurate facility.  I see while I was composing this Bruce Leban came
up with a very plausible convention, too.

 > Use cases are mostly bug-free refactoring,

But the refactoring is very "meta" in the sense that you're creating a
string that refers to the name of an object.  You either need an
external agent or computation which would be confused if the name were
incorrect, or to be using eval() for this to be an actual refactoring.
All of this seems very "code smelly" to me.  If you're refactoring
this automatically, you're doing it wrong. ;-)

I don't see any use case that f-string '=' doesn't satisfy well enough
to make a new builtin pseudo-function[1] justifiable.

 > E.g. would it be faster than `Class.__name__`?

You're working with strings expressing code.  If it's about speed,
there's got to be a better way.



Footnotes: 
[1]  In C#, 'nameof' is first looked up as a function reference, and
if found that is called.  Otherwise the compiler macro is used.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XC7QRRJN7DSF2AKDCMZWANQVOPJM65QI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extract variable name from itself

[Stephen J. Turnbull]

Jonathan Fine writes:

 > We can also use locals() to 'inverse search' to get the name, much
 > as in the original post.

As has already been explained, locals() (and any namespace for that
matter) is a many-one mapping, and therefore the inverse is not
well-defined.

At least for the 'print(f"count is {count}")' example, you really need
the compiler's help to get it right, unless you are willing to do it
the other way around:

def debug_name(name: str) -> None:
print(f"{name} is {eval(name)}")

but in general that's fraught with all the problems of using eval().

Steve



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TIDEI74KQMEUCORSB7XMTBITDFYSK7D7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Revise using the colon ( : )

[Stephen J. Turnbull]

Celelibi writes:
 > I just want to mention that without colons, one-liners could become
 > ambiguous:

I have no sympathy for optional colons.  No colons, no one-liners.

(Of course optional colons are a great strategy for trying it out, but
in your test programs you should maintain no-colon discipline.)

STeve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B46LYKE7W5XHDUDTFA75OI2NIR25ISWI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Revise using the colon ( : )

[Stephen J. Turnbull]

Morteza Mirzakhan writes:

 > In my opinion, using a colon after keywords such as `if`, `elif`,
 > `else`, `for`, `while`, `try`, `except`, and `finally` is quite
 > unnecessary.

Sure, but that would be a different language.  I like the formality of
the colon, and it helps in visually parsing the test in cases where
the test expression covers more than one line and contains
parentheses.  I think you're very unlikely to get any uptake from the
core developers on this, not because I don't like it but because it's
a change for no reason except one person doesn't like colons.

But it should be an easy experiment to try.  Change the grammar so
that colons are optional in those places.  Then try to build Python
and see what happens.  I think that has a good chance of working.  (It
definitely won't work if you omit the colons entirely: all of the
Python code in the stdlib will stop working!)

If it does, then you can try your "Pytho" language (missing one
character at the end, you see?) and see how you like it and if you can
get uptake from others.  If you do, then you can prohibit colons in
those places and start the tedious work of making the stdlib work
without them.  I still doubt you'll get a large following from
Pythonistas but who knows? :-)



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AWD6QBGRG4GU6OZ2ENJTWEG3E3S3ZDZE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "Curated" package repo?

[Stephen J. Turnbull]

George Fischhof writes:

[For heaven's sake, trim!  You expressed your ideas very clearly, the
quote adds little to them.]

 > it has got to my mind that even just grouping similar / same goal
 > packages could help the current situation.

This is a good idea.  I doubt it reduces the problem compared to the
review site or the curation very much: some poor rodent(s) still gotta
put the dinger on the feline.

However, in designing those pages, we could explicitly ask for names
of similar packages and recommendations for use cases where an
alternative package might be preferred, and provide links to the
review pages for those packages that are mentioned in the response.
We can also provide suggestions based on comparisons other users have
made.  (Hopefully there won't be too many comparisons like "this
package is the numpy of its category" -- that's hard to parse!)

 > Additionally perhaps the users could give relative valuation,

Not sure asking for rankings is a great idea, globally valid rankings
are rare -- ask any heavy numpy user who occasionally uses the sum
builtin on lists.

 > for example there are A, B, C, D similar packages, users could say:
 > I tried out A and B, and found that A is better then B, and could
 > have some valuation categories: simple, easy, powerful etc. This
 > would show for example that package A is simple, but B is more
 > powerful

These tags would be useful.  I think the explanation needs to be
considered carefully, because absolutes don't really exist, and if
you're comparing to the class, you want to know which packages the
reviewer is comparing to.  I'm not sure many users would go to the
trouble of providing full rankings, even for the packages they've
mentioned.  Worth a try though!

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MGYK3EEEYKISRQMFJY7U73ID6VBQY4AT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal for get_or function in Python dictionaries

[Stephen J. Turnbull]

Chris Angelico writes:
 > On Thu, 20 Jul 2023 at 15:33, Stephen J. Turnbull
 >  wrote:

C'mon Chris, "that was then, this is now".  Catch up, I've changed my
position. ;-)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LDY22EATTWYUHS53ERHM4PKK6GTIE5G2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal for get_or function in Python dictionaries

[Stephen J. Turnbull]

Dom Grigonis writes:

 > The thing I am not content with (to make it common practice in my
 > coding style) is inconsistencies between a) order of terms
 > and b) functionality, of statements and their analogous
 > expressions.

I think that's reasonable.  But I don't think Python is the language
for that.  There are too much existing contrasts, such as loop vs.
comprehension and conditional statement vs. ternary expression.  Guido
deliberately chose to vary those expression syntaxes from their
statement equivalents.  I don't think there's any ambiguity if you say
that wrapping a compound statement in parentheses creates an
expression with restricted syntax (namely, the statements the syntax
controls become expressions):

(if condition_1: expression_1
 elif condition_2: expression_2
 else: expression_else)

[for element in iterable: if condition: expression(element)]

But Guido (and I think everybody else) thought "N! not *that*!"

 > ??
 >  `Deferred evaluation`, if was to gain traction in a similar manner
 >  as e.g. `annotations` are now experiencing, would cover all of the
 >  cases I was looking at & more.

As Chris was saying earlier, it's at minimum going to take some genius
creativity to "cover all and more", because a closer look at what
people mean by "deferred" shows that in different cases it is
semantically different.  In particular, there are a number of choices
that Chris made in PEP 671 that aren't compatible with several of the
proposals for Deferred objectss, while Deferreds can't give some of
the benefits of the PEP.

I didn't like that PEP then; I was in the camp of "let's get genuine
Deferreds, and use them to cover some of the use cases for PEP 671."
I don't actively want the PEP now.  def-time evaluation of defaults
doesn't catch me and I haven't had trouble teaching it.  The
"if arg is None: arg = Mutable()" idiom is rare enough that I prefer
it to adding syntax to the already complex function prototype.

But if you think it might be a good idea, I encourage you to take a
close look.  For some use cases it's definitely an improvement, and at
least I won't vocally oppose the PEP now -- haven't seen any progress
on "true Deferreds".  Perhaps others' opposition has softened, too.
Adding another proponent is another way to help get it going again.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R5RDLOSEBPEQQ5BLECV7HPKZAX6HYSM2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal for get_or function in Python dictionaries

[Stephen J. Turnbull]

Dom Grigonis writes:

 > > But "encourages one-liners" is generally considered an
 > > anti-pattern in Python.  Let's see why,

 > Here I am a bit confused, how is this the case in the language
 > containing list comprehensions?

I don't think of list comprehensions in terms of line count.  I often
write multiple-line list comprehensions and genexps.  Especially when
nested or with an 'if' clause,, I often use multiple lines even though
the whole thing would fit on a single line because I feel it expresses
the structure better.  Comprehensions and genexps are an especially
nice context for that, because they are always equipped with
parentheses, so you are not constrained by the usual rules of
indentation, and don't need line continuing backslashes.

Also consider

for x in list1: list2.append(foo(x))

list2.extend([foo(x) for x in list1])

The second form is one character less concise than the former, yet far
more expressive.  Once genexps were introduced, we could make it one
character more concise than the for loop with

list2.extend(foo(x) for x in list1)

but it's unclear that this is an improvement over the list
comprehension.  (That's probably due to the facts that I don't use
genexps that often, and that I think of list.extend as a concatenation
of lists even though it's documented as taking an iterable.  If most
people have more facility with genexps than I do, it's a small but
clear improvement.)

 > I would understand if it was swapped with “bad quality & unreadable
 > 1-liners”.

That dodges the question of when does a one-liner cross over to "bad
quality and unreadable", though.

 > Also, I would rephrase “encourage 1-liners” to “promote readable
 > and expressive structures that are balanced in their brevity versus
 > complexity”.
 > I am not encouraging 1-liners,

Well, the word "one-liner" has a history.  "One-liner" is a category
by that name in the Obfuscated C Contest, and Perl programmers often
take pride in how much they can accomplish on the command line,
without starting an editor or interactive interpreter.  Some of us
old-timers are going to take it the wrong way.  I'm not sure if less
indoctrinated people would take it to mean "readable and expressive
structures that are balanced in their brevity versus complexity". :-)

In any case, whether you intend it or not, making the ternary
expression more terse would encourage examples of the kind you
present.

 > I am more arguing that certain things in relation to average
 > complexity should take no more than 1-line. I am always very happy
 > to write multiple lines.

A Python ternary expression usually takes only part of one line.  They
allow you to put a simple conditional in the middle of a longer
expression.  However, they're not amenable to nesting ternaries,
except in very special circumstances.  I think that's a good thing,
you don't.  We can both be right, you know!  I'm just trying to
explain why I think that way, and making the claim (which may be
incorrect) that the Pythonistas who influence language design do, too.

 > Btw, here I would probably prefer:
 > def clamp_int(n: int, lo: int, hi: int):
 > if lo > hi:
 > raise ValueError(f'{lo=} > {hi=}')
 > return lo <= n <= hi ? n : (n > hi ? hi : lo)

To my eye, that's a great example of "concise but not expressive".
Parsing it requires reading almost character by character, the nuance
of the conditional changes from "True => usual case" to a genuine
choice, and verifying the correctness of the expression is nontrivial
compared to verifying the if statement form.

 > I think the place I am coming from is more about balance than
 > brevity.

OK.  But for me that's hard to see when it's expressed by counting
lines.

Balance and nuance sometimes can be expressed in words, but in cases
where the line is hard to draw, we often go through a large corpus and
find as many examples as possible and compare the existing code with
versions using the new syntax.  Typically it's the stdlib, but since
you mention numerical analysis, numpy or something based on it like
pandas might provide better examples for you.  I don't recommend that
here.  The existence of a semantically equivalent ternary expression
already makes that an impossible lift.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KLOYV2HIKJHAEJT7VCYAGYL3DUOZRGZS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal for get_or function in Python dictionaries

[Stephen J. Turnbull]

Dom Grigonis writes:

 > I came to this, because people seem to want one-liners for certain
 > things and what I came up with is that maybe more concise if-else
 > expression could help.

But "encourages one-liners" is generally considered an anti-pattern in
Python.  Let's see why,

 > # Fairly reasonable case.

 > def foo(a: bool, c: float, d: float)
 > val = a ? (c ? c : d) : (d ? d : c)
 > return val

What's not so great here?

1.  The argument names are not going to be single characters, unless
you intentionally name that way for the sake of one-line-ism.
That severely detracts from your claim of readability.  The
one-liner is arguably[1] more readable than the Python version,
but you've definitely made the whole function less readable.

2.  Comparing floats, even against zero, is a bad idea.  So I would
wrap them in math.isclose:

val = a ? (not isclose(c, 0) ? c : d) : (not isclose(d, 0) ? d : c)

Pretty long, not that easy to read.  Of course if you had
intelligible variable names it would be worse.

The point is *not* to give you a hard time about comparing floats
for equality, we've all done that.  It's that in even in this
example, done safely you have more complex expressions than just
variable references.  In general you will have function calls
adding parentheses -- and thus confusion.  Perhaps there will be
lists or tuples involved.  You can precompute them and assign
them to short name variables, but then you lose the one-liner-ness.

3.  OK, how about ints then?  Yes, that would work, but how frequently
are you going to be comparing against 0?  Here's a more realistic
case, with readable short argument names:

def clamp_int(n: int, lo: int, hi: int):
# I wouldn't elide this test because if lo > hi you get a
# plausible but necessarily erroneous value
if lo > hi:
raise ValueError(f'lo = {lo} > {hi} = hi')
val = n < lo ? lo : (n > hi ? hi : n)
return val

Agreed, the expression using Python's syntax would be worse, but I
think this is much more readable:

def clamp_int(n: int, lo: int, hi: int):
if lo > hi:
raise ValueError(f'lo = {lo} > {hi} = hi')
if n < lo:
val = lo
elif n > hi:
val = hi
else:
val = n
return val

and it would be more readable yet if you got rid of the
assignments and just returned the value as soon as you see it:

def clamp_int(n: int, lo: int, hi: int):
if lo > hi:
raise ValueError(f'lo = {lo} > {hi} = hi')
# Hi, David!  Default first!!
if lo <= n <= hi:# Yes, Virginia, valid Python!
return n
elif n > hi:
return hi
else:
return lo

(Yes, I know that some folks argue that suites should have one
entry point and one exit point, but I don't think it's a problem
here because of the extreme symmetry and simplicity of the
conditional.  IMHO YMMV of course)

 > I dare you to do a 1-liner with current if-else.

I do use ternary expressions occasionally, but almost never nested.
Writing one-liners is never a goal for me, in part because my old eyes
can't take in more than 35-40 characters in a gulp.

 > So maybe, just maybe, making already existing expression more
 > readable can also be a valid suggestion?

We *won't do that*, because of backward compatibility.  The syntax of
an exist expression cannot change without invalidating a lot of
existing code.  That's one reason why the bar to new syntax is so
high, and it took so long to get the ternary expression: you really
want to get it right.  That's why Python (like C!) prefers to add to
the stdlib rather than the language.

Now, you can add a new one that does the same thing, and that's been
done.  IIRC it took a while to get +=, and C-like ++/-- increment
operators have been requested over and over again.  AFAIK the async
syntaxes do nothing that can't be done with generators, but they make
it a lot easier to do it right in the most common cases.

 > As I said, in my opinion it would solve many existing queries and
 > requests, just because certain things would become very pleasant
 > and obvious how to do in simple one-liners.

One-liners are almost by definition less readable.  Even in math
papers we break equations into a series of definitions and then build
up the final equation from there.  As M. Spivak put it in *Calculus on
Manifolds*,

Stokes' theorem shares three important attributes with many fully
evolved major theorems:
1.  It is trivial.
2.  It is trivial because the terms appearing in it have been
properly defined.
3.  It has significant consequences.

When I have the time and knowledge, I aspire to programming that
way. :-)  YMMV

 > Simpler and more logically convenient if-else combined with other
 > elegant python expressions would 

[Python-ideas] Re: Conditional 1-line expression in python

[Stephen J. Turnbull]

David Mertz, Ph.D. writes:

 > I think the Python version does the right thing by emphasizing the
 > DEFAULT by putting it first, and leaving the predicate and fallback
 > until later in the expression (right for Pythonic code, not right
 > for other languages necessarily).

Aside: I think that's true for Pythonic scripting, but in general I
think you're just as likely to find that it's a genuine choice, and
that the condition is as interesting as either choice.

Variation on the theme of conditional expression: I'd often like to
write

var = computed() if cond() else break

or

var = computed() if cond() else continue

I wonder if that syntax has legs, or if it's too cute to stand.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WUSUIRQ5XLFSS5JQKI3DYZ7SIDABERKY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "Curated" package repo?

[Stephen J. Turnbull]

James Addison via Python-ideas writes:

 > The implementation of such a system could either be centralized or
 > distributed; the trust signals that human users infer from it
 > should always be distributed.

ISTM the primary use cases advanced here have been for "naive" users.
Likely they won't be in a position to decide whether they trust Guido
van Rossum or Egg Rando more.  So in practice they'll often want to go
with some kind of publicly weighted average of scores.

To avoid the problem of ballot-box stuffing, you could go the way that
pro sports often do for their All-Star teams: have one vote by anybody
who cares to register an ID, and another by verified committers,
including committers from "trusted" projects as well.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4XKWJDHQWBCX7HIX7UT5GJNXMFOLMDWY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Suspected Spam]"Curated" package repo?

[Stephen J. Turnbull]

Chris Angelico writes:

 > Part of the desired protection is the prevention of typosquatting.
 > That means there has to be something that you can point pip to and
 > say "install this package", and it's unable to install any
 > non-curated package.

I think that the goalposts are walking though.  How do you keep
non-curated packages out of requirements.txt?  Only if you have a
closed ecosystem.  Sounds like Anaconda or Condaforge or Debian to me,
and people who want such a closed system should pick one-- and
preferably only one --to support.

The basic request as I understood it was to reduce what Chris Barker
characterized as the cost of sifting through a maze of twisty little
packages all alike, except that some are good, and some are bad, and
some are downright ugly.  Part of that is indeed to avoid typo-
squatting malware.  However, most of the squatters I'm aware of use
names that look like improved or updated versions, and would not be
frequently typoed.  So my "click through to PyPI" approach would
filter a majority, possibly a large majority, of non-curated packages.

If people really want this somewhat draconian restriction to curated
packages, fine by me (I'll stick to proofreading requirements.txt very
carefully plus pip'ing from PyPI myself).  I just don't see how it
works or has advantages over existing options.

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VYKKDCVPBESMKHVD2ORSDSPNULRVPIGW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "Curated" package repo?

[Stephen J. Turnbull]

Christopher Barker writes:

 > Yes, it needs to be funded somehow, but some sort of donation / non
 > profit / etc funding mechanism would be best -- but I don't think
 > peer reviewers should be paid. Peer review in academic journals
 > isn't cash compensated either.

It's been done.  The most common scheme is nominal compensation (say
USD50 per review) dependent on beating a relatively short deadline
(typically 1-3 months).  But this is not really the same as academic
publishing.  It's also not the same as movie and book reviewers who
are paid staffers (at least they used to be in the days of paper
journals).  It has aspects of both.  It might work here, although
funding and appointment of reviewers are tough issues.

 > I had to look that up: "Decentralized autonomous organization (DAO)"
 > 
 > So, yes.

Please, no.  DAOs are fine when only money is at risk (too risky for
me, though).  But they're a terrible way to manage a community or its
money.  Too fragile, too inflexible.  The history of DAOs is basically
an empirical confirmation of Arrow's Impossibility Theorem.
https://simple.wikipedia.org/wiki/Arrow%27s_impossibility_theorem

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZLBEHQGMFIA5PR26XVDQF4YAVPIOYWY4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] [Suspected Spam]"Curated" package repo?

[Stephen J. Turnbull]

Christopher Barker writes:

 > So the odds that there's a package that does what you need are
 > good, but it can be pretty hard to find them sometimes -- and can
 > be a fair bit of work to sift through to find the good ones -- and
 > many folks don't feel qualified to do so.

"Fair bit of work sifting" vs. "very hard work writing your own of
higher quality" sounds like a VeryGoodDeal[tm] to me.

As for "unqualified", if it's OK to be writing a program where you're
unqualified to evaluate dependency packages, maybe it really doesn't
matter if the package is best of breed?  There are lots of programs
where it doesn't matter, obviously.  But if there's problem, it won't
be solved by curation -- even best of breed is liable to be misused.

 > 4)  A self contained repository of packages that you could point
 > pip to -- it would contain only the packages that had met some
 > sort of "vetting" criteria. In theory, anyone could run it, but
 > a stamp of approval from the PSF would make it far more
 > acceptable to people. This would be a LOT of work to get set
 > up, and still a lot of work to maintain.

Why "self-contained"?  I always enter PyPI through the top page.  I'd
just substitute curated-pypi.org's top page.  Its search results would
be restricted to (or prioritize) the curated set, but it would take
me to the PyPI page of the recommended package.

Why duplicate those pages?  Would you also duplicate the storage?  The
only reason I can imagine for doing a "deep copy" would be to avoid
the accusation of piggybacking on PyPI's and PyPA's hard work.  Even
if maintaining separate storage, many developers who use it would
still depend on pip -- which gives priority to PyPI.  They'd use the
curated site to choose a package, but then just write its name in
requirements.txt -- and of course that would work.  So you don't even
really avoid depending on PyPI for bandwidth (of course PyPI is going
to spend on storage, anyway, so that doesn't count).

 > Personally, I think (4) is the best end result, but probably the
 > most work as well[*], so ???

Setup is pretty straightforward, maybe expensive if you go the
self-contained route.  But all the extra ongoing work is curation.
And that's *really* hard to sustain.

Riffing on "curated-pypi.org", I think it would be difficult to get
"curated DOT pypi.org" for the reasons that have been repeatedly
advanced, but why not your_team_name_here.curated.pypi.org?  Set up a
Patreon and get the TechDirt guy or somebody like that to write a
monthly column reviewing the curators (once you get a lot, call it
"featured curators", probably with the excuse that they specialize in
some application field) or soliciting curators as the curated.pypi.org
page.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZE4ZO4HZFJJCLAD3OOVMLRWI3WW6BLGD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dict method to retrieve a key from a value

[Stephen J. Turnbull]

MRAB writes:

 > I would hope that if the maintainer knew that the package was on
 > the curated list, they would let the list know if they were going
 > to stop maintaining it. That would be part of being on the list -
 > informing the list's maintainers of any significant change of
 > status.

How do you enforce that that?  Yup, it's the responsibility of the
list curators because the package maintainers aren't always going to
do it.  Really, curating has to include removing packages that no
longer deserve listing as well as adding ones that do.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZATQB3NBC2CJFYBHV3TXZ3624HMDKSOF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dict method to retrieve a key from a value

[Stephen J. Turnbull]

Chris Angelico writes:
 > On Sat, 1 Jul 2023 at 18:43, Stephen J. Turnbull
 >  wrote:

 > > But if somebody's going to put in effort to review PyPI, I'd
 > > really rather see them go after "typo squatters".
 > 
 > They're definitely working on that.

OK, I'll take that as "Steve, report the ones you've found
already". :-)  (Since they're a project I work on I feel like I should
give an expert opinion, rather than just a "I smell fish" email. :-)

 > Yeah. All we need is for people to start ranting on the internet.
 > Can't be THAT hard right?

Good rants are hard to find!

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GOMLHR56YOSQ6VEFGH7NSY3TIZT6JESD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dict method to retrieve a key from a value

[Stephen J. Turnbull]

Chris Angelico writes:
 > On Sat, 1 Jul 2023 at 01:15, Christopher Barker
 >  wrote:

 > > Totally different topic, but I do think that a "curated" package
 > > repo would be helpful -- there is a lot of cruft on PyPi :-(

Sounds like a "standard library".  I understand the difference, but
Like Chris A I'm dubious about whether there's really a lane for it,
or whether like bike lanes in Japan you'd just find a lot of illegal
parking in it. ;-)

But if somebody's going to put in effort to review PyPI, I'd really
rather see them go after "typo squatters".  Most are probably just
clout chasers, but we know that some are malware, far more dangerous
than merely "cruft".

Chris Angelico writes:

 > Instead, what I'd like to see is: Personal, individual blogs,
 > recommending packages that the author knows about and can give
 > genuine advice about.

I think this is a good way to go, expecially if reviewers link to each
other, building community as well as providing package reviews.  For
example, while my needs are limited enough that I haven't actually
tried any of his stuff, I've found Simon Willison's (datasette.io)
tweetqs interesting.  (datasette itself, of course, and he's tweeted a
lot about LLMs recently too, but here I'm referring to his more random
tweets about utilities he's discovered or created.)

There are also some idiosyncratic curated package collections (the
FLUFL packages, etc), as well as a lot of frameworks (Django, the Zope
components, lazr) that seem to sprout utility packages regularly.  I'm
sure there's a lane if somebody wants to go around blogging about all
the "stuff" they see.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MPN4STTPDRHMDJWEVFWC3ZUFX6R3QJGA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dict method to retrieve a key from a value

[Stephen J. Turnbull]

Christopher Barker writes:

 > > If O(log N) is good enough and bijectivity is guaranteed by some
 > > other mechanism, a bisection search on d.items() with
 > > key=lambda x: x[1] does the trick.

 > You'd also have to keep it sorted by value.

I assumed you can do that with OrderedDict.  But yeah, it's a little
more complex.  Maybe some of these ideas are complex and useful enough
to deserve PyPI implementations, and if they prove to have corner
cases and get take up, then consideration for stdlib.

 > So now you are guaranteeing bijectivity and keeping it sorted --
 > I'd just use two dicts :-) Though then the values would have to be
 > hashable, so there's that.

Right (and I missed that throughout, so tyvm).  So much as the
"generic" theory of "low O" "bijection" is attractive, the
implementation details are application-specific.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2YV4C22IMTO5A7E3FJTZZ5QU2L7YPJAB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] dict method to retrieve a key from a value

[Stephen J. Turnbull]

Andre Delfino writes:

 > A dict method to retrieve the key of a value from a bijective dict
 > would have come in handy to me in several occasions:
[...]
 > I do understand that retrieval wouldn't be O(1).

If O(log N) is good enough and bijectivity is guaranteed by some other
mechanism, a bisection search on d.items() with key=lambda x: x[1]
does the trick.

If you know the dict is supposed to be one-to-one and the keys and
values are disjoint sets, as in your example, just

def bijective_add(d, k, v):
if k in d and d[k] != v:
raise BijectiveDictValueChangeError
d[k] = v
d[v] = k

gives O(1) both ways.  Maybe you need some kind of check to be sure
you're retrieving the right type for the calling code.

Otherwise you can pair two dicts in a class and endow it with your
inverse method.  I would do the check for bijectivity on addition (as
above) rather than on retrieval, though.

If you really want to save the space, you can add an additional
hashtable for the values in a C module, but it's not clear to me that
any particular choice for bijectivity checks would be universally
desirable in applications so that seems premature.

So I think you need to make clear what your goals are since there are
at least four solutions with varying performance characteristics.
I'm agnostic on whether a dict type which guarantees bijectivity would
be a good addition.  The mathematician in me wants it, but my
experience says the dict pair is good enough, YAGNI (for values of
"you" == "me", anyway).

Regards,
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5EEE27MQZLFYF3HJMFVR3KINPIZB3HRI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] `join` method for the `list` class ... `list.join`

[Stephen J. Turnbull]

Samuel Muldoon writes:

 > Python's `str` class has a method named `join`
 > 
 > I was wondering if a future release of python could have a `list.join`
 > which behaves in a similar fashion.
 > 
 > result = [99].join([1, 2, 3])
 > print(result)
 > # prints [1, 99, 2, 99, 3]

I wouldn't call that an example of .join.  To me, .join takes an
argument which in the most general case is iterable of iterables.
Your example's argument is the more general iterable of object.  So I
don't think .join is a good name for this function.  Maybe
.interpolate, although that has a different meaning in statistics.

Also, .join "works" for str because it's *not* general.  It's useful
because the "algebra" of strings is built of making lists of strings
and then "flattening" them into a single string.  Encoding the single
string so that it can be "decoded" into the original list of words
involves inserting separator characters such as space, newline, crlf,
or comma.  And that's exactly .join!

I can't think offhand how I would use either a generalized .join or
this function, except with str or bytes, both of which already have
.join.  Do you have an application for either a list join or for this
interpolation function?  .join might be useful in a signal processing
application for the same reason it's useful for strings, but the
interpolation semantics, I don't see it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5ZWQEPN3LIZS5NGCR73RWOKVMTXUSQ3L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Ampersand operator for strings

[Stephen J. Turnbull]

Rob Cliffe writes:

 > Perhaps where you're not laying out a table,

I'm an economist, laying out tables is what I do. :-)  Getting
serious:

 > but constructing a  human-readable string?  So
 >      s1 + ' ' + s2 + ' ' + s3
 > or
 >      ' '.join((s1, s3, s3))
 > would become
 >      s1 & s2 & s3
 > saving you a bit of typing.  Just sayin'.

str '+' has long been quite rare in my coding.  str concatenation is
almost never in an inner loop, or slighly more complex formatting is
the point.  f-strings and .format save you the type conversion to str.
So I don't find that occasional saving at all interesting.  A
vanishingly small number of my str constructions involve only strs
with trivial formatting.

What interests me about the proposal is the space-collapsing part,
which a naive f-string would do incorrectly if, say, s2 == '' or s3 ==
'\t\t\ttabs to the left of me'.  But where does this space-surrounded
str data come from?

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HTEPBAI6EKXTUSU5SYZDZJUCOWHQN6C7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Ampersand operator for strings

[Stephen J. Turnbull]

Steven D'Aprano writes:
 > I like the look of the & operator for concatenation, so I want to like 
 > this proposal. But I think I will need to see real world code to 
 > understand when it would be useful.

I have to second that motion.  Pretty much any time I'm constructing
lines containing variable text, each string value arrives stripped and
I far more often want padding of variable width values rather than
space compression.

I admit that I use M-SPC (aka just-one-space) lot in Emacsen, but I
can't recall wanting it in a program in any language.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DZXQJ4GQTDEEMOO6VX4VJE45ZN7XNMR4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Multiple arguments to str.partition and bytes.partition

[Stephen J. Turnbull]

James Addison via Python-ideas writes:
 > On Sun, 8 Jan 2023 at 08:32, Stephen J. Turnbull
 >  wrote:

 > Trying to avoid the usual discussions about permissive parsing /
 > supporting various implementations in-the-wild: long-term, the least
 > ambiguous and most computationally-efficient environment would
 > probably want to reduce special cases like that?  (both in-data and
 > in-code)

That's not very human-friendly, though.  Push that to extremes and you
get XML.  "Nobody expects the XML Validators!"

 > Structural pattern matching _seems_ like it could correspond here, in
 > terms of selecting appropriate arguments -- but it is, as I understand
 > it, limited to at-most-one wildcard pattern per match (by sensible
 > design).

If I understand what you mean by "structural pattern matching", that
seems more appropriate to parsing already tokenized input.

 > I suppose an analysis (that I don't have the ability to perform
 > easily) could be to determine how many regular expression codesites
 > could be migrated compatibly and beneficially by using
 > multiple-partition-arguments.

My guess is that for re.match (or re.search) it would be relatively
few.  People tend to reach for regular expression matching when they
have repetition or alternatives that they want to capture in a single
expression, and that is generally not going to be easy to capture with
str.partition.  But I bet that *many* calls to re.split take regular
expressions of the form f'[{separators}]' which would be easy enough
to search for.  That's where you could reduce the number of regexps.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MLTLIMCLCRQDCSHTJWO5GBNPBEID6N4H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Multiple arguments to str.partition and bytes.partition

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > I mean, if all you are doing is splitting the source by some separators 
 > regardless of order, surely this does the same job and is *vastly* more 
 > obvious?
 > 
 > >>> re.split(r'[:;]', 'foo:bar;baz')
 > ['foo', 'bar', 'baz']

"Obvious" yes, but it's also easy to invest that call with semantics
(eg, "just three segments because that's the allowed syntax") that it
doesn't possess.  You haven't stated how many elements it should be
split into, nor whether the separator characters are permitted in
components, nor whether this component is the whole input and this
regexp defines the whole syntax.  The point of the "well-known idiom"
is to specify most of that (and it doesn't take much much more to
specify all of it, specifying "no separators in components" is the
most space-consuming part of the expression!)

Your other alternatives have the same potential issues.

 > > But that's characteristic of many examples.
 > 
 > Great. Then for *those* structured examples you can happily write your 
 > regex and put the separators in the order you expect.
 > 
 > But I'm talking about *unstructured* examples where you don't know the 
 > order of the separators, you want to split on whichever one comes first 
 > regardless of the order, and you need to know which separator that
 > was.

That's easy enough to do with a (relatively unknown to some ;-)
regular expression:

re.match("([^;:]*)([;:])(.*)", source)

The question is whether the need is frequent enough and that's hard
enough to understand / ugly enough to warrant another method or an
incompatible extension to str.partition (and str.rpartition).[1]

 > > Examples where the order of separators doesn't matter?  In most of the
 > > examples I need, swapping order is a parse error.
 > 
 > Okay, then you *mostly* don't need this.

I already knew that.  Without real examples, I can't judge whether I'm
pro-status quo or pro-serving-the-nonuniversal-but-still-useful-case.

 > str.partition does *one* three way split, into (head, sep, tail).
 > If you want to continue to partition the tail, you have to call
 > it again.

I'm much more favorable to proposals where str.partition and
str.rpartition split at *one* point, but the OP seemed intended to do
more work (but not arbitrary amounts!) per call.

 > I'm not sure I quite understand you there, but if I do, I would
 > prefer to split the string and then validate the head and tail
 > afterwards, rather than just have the regex fail.

For me, often that depends on how hard I'm willing to work to support
users.  If the only user is myself, that's very often zero.  In the
case of the "well-known idiom", the only ways the regexp can fail
involve wrong number of separators.  I'd be willing to impose that
burden on users with a "wrong number of separators" message.  Another
case is where I want an efficient parser for the vast majority of
conformant cases and am willing to do redundant work for the error
cases.



Footnotes: 
[1]  Here "incompatible" means that people writing code that must
support previous versions of Python can't use it.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WG57TUZEEPD73QNFNCRR2LOA5NEZFP4J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Multiple arguments to str.partition and bytes.partition

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > On Sat, Jan 07, 2023 at 10:48:48AM -0800, Peter Ludemann wrote:
 > > You can get almost the same result using pattern matching. For example, 
 > > your
 > > "foo:bar;baz".partition(":", ";")
 > > can be done by a well-known matching idiom:
 > > re.match(r'([^:]*):([^;]*);(.*)', 'foo:bar;baz').groups()
 > 
 > "Well-known" he says :-)

It *is* well-known to those who know.  Just because you don't like
regex doesn't mean it's not well-known.  I wouldn't use that idiom
though; I'd use an explicit character class in most cases I encounter.

 > I think that the regex solution is also wrong because it requires you 
 > to know *exactly* what order the separators are found in the source 
 > string.

But that's characteristic of many examples.  In "structured" mail
headers like Content-Type, you want the separators to come in the
order ':', '=', ';'.  In a URI scheme with an authority component, you
want them in the order '@', ':'.  Except that you don't, in both those
examples.  In Content-Type, the '=' is optional, and there may be
multiple ';'.  In authority, the existing ':' is optional, and there's
an optional ':' to separate password from username before the '@'.

And it gets worse: in the authority case, the username is optional.
In the common case of anonymous access, the username is omitted, so

user, _, domain = "example.com".partition('@')

does the wrong thing!

 > If we swap the semi-colon and the colon in the source, but not 
 > the pattern, the idiom fails:
 > 
 > >>> re.match(r'([^:]*):([^;]*);(.*)', 'foo;bar:baz').groups()
 > Traceback (most recent call last):
 >   File "", line 1, in 
 > AttributeError: 'NoneType' object has no attribute 'groups'
 > 
 > So that makes it useless for the case where you want to split of any of 
 > a number of separators, but don't know which order they occur in.

Examples where the order of separators doesn't matter?  In most of the
examples I need, swapping order is a parse error.

 > You call it "almost the same result" but it is nothing like the result 
 > from partition. The separators are lost,

Trivial to fix, just add parens, in the simpler grouping form as a
bonus!  I'm not asking you to like the resulting regexp better, just
pointing out that your dislike of regex is driving the discussion in
unprofitable directions.

 > and it splits the string all at once instead of one split per call.

So does the original proposal, that's part of the point of it, I
think.

I really don't see any of the variations on the proposal as a
particularly valuable addition.  It's already easy to screw up your
parse with str.partition (the authority example: although you can fix
the order problem with '@' by using str.rpartition, the multiple
optional ':' mean that whichever r?partition you use, you can get it
wrong unless you check the order of '@' and ':', so you have to use a
recursive parse, not a sequential parse).  But you can write a regex
version of authority to give a sequence of tokens rather than a parse,
and you convert that into a parse by checking each element of the
sequence for None in a deterministic order.  I prefer the latter
approach (Emacs user since Emacs was programmed in TECO), but as long
as you allow me to use regex for character classes and sequences, I
can live with retrictions on use of regex in the style guide.

Parsing is hard.  Both regex and r?partition are best used as low-
level tools for tokenizing, and you're asking for trouble if you try
to use them for parsing past a certain point.  My breaking point for
regex is somewhere around the authority example, but I wouldn't push
back if my project's style guide said to to break that up.  I *would*
however often prefer regexp to r?partition because it would allow
character classes, and in most of the areas I work with (mail, URIs,
encodings) being able to detect lexical errors by using character
classes is helpful.  And I would prefer "one bite per call" partition
to a partition at multiple points.  Where I'm being pretty fuzzy, the
.split methods are fine.

-- Yet another Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VDZQVHGUPAOUCPL4HPAXFTQPNAHNJZIK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

[Stephen J. Turnbull]

Chris Angelico writes:

 > I don't think str.upper() is the place for it; Python has a locale
 > module that is a better fit for this.

Many would argue that (POSIX) locales aren't a good fit for
anything. :-)

I agree that it's kind of hard to see anything more complex than a
fixed table for the entire Unicode repertoire belonging in str,
though.  (I admit that my feeling toward Erdogan makes me less
sympathetic to the Turks. :-)  Use of locale notation in keys for more
sophisticated treatment is hard to beat as far as I know, though.

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7KTN6H7HKXJL7NI3MWEQ7ZY6V47XUVJQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

[Stephen J. Turnbull]

Christopher Barker writes:

 > But collections.UserString does exist -- so if you want to subclass, and
 > performance isn't critical, then use that. Steven A pointed out that
 > UserStrings are not instances of str though. I think THAT is a bug.

I guess, although surely the authors of that class thought about it.

Anyway, this could probably be improved with a StringLike ABC (and we
get to bikeshed whether bytes and bytecode are StringLike -- see ya,
I'm outttahere!)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2M3UWBJDFOROYURIAWTIZ23WRVLIWHHG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extending LiteralString or Literal with structural validation

[Stephen J. Turnbull]

Ricky Teachey writes:

 > This isn't to say it should definitely be added to the language, that's a
 > tough hurdle. But boy would I have used it.

IIUC, Mathew's idea doesn't need to be added to *the* language (the
one defined by the Language Reference).  It needs to be added to the
languages used by type checkers.

So if you want this and know a type checker author, send them pizza! :-)

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7DRE464OVRN5EAA626N5TGRUMHD2OJNP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

[Stephen J. Turnbull]

Brendan Barnwell writes:

 >  What it means for me for something to "be an HTML string" (or more 
 > precisely, to be an instance of HTMLString or whatever the class name 
 > is) is for it to be a string that has an extra tag attached to the 
 > object that means "this is HTML".

I don't like tags that lie.  Seems pointless (see below).

 > The point is that overrides are for specifying the *new* behavior
 > of the subclass (i.e., not allowing certain slice operations); you
 > shouldn't have to override methods just to retain the superclass
 > behavior.

Do you mean "retain the subclass behavior" here?  AFAICS what's being
called "hostile" is precisely retaining *superclass* behavior.

 >  I mean, we were talking about this in the context of syntax 
 > highlighting.  The utility of HTML-string highlighting would be 
 > seriously reduced if only *valid* HTML could be in an HTML string.

The proposed HTMLstring *class* is irrelevant to syntax highlighting,
regardless of its functionality.  The OP (and his syntax-highlighting
text editor!) wants standard literal syntax *in source code* that
allows an editor-that-is-not-as-programmable-as-emacs-or-vim to
recognize a fragment of text (typically in a literal string) that is
supposed to be highlighted as HTML.  Syntax highlighting is not aided
by an HTMLstring object in the *running Python program*.

I really don't understand what value your HTMLstring as str + tag
provides to the OP, or to a Python program.  I guess that an editor
written in Python could manipulate a list of TaggedString objects,
but this is a pretty impoverished model.  Emacsen have had extents/
overlays since 1990 or so, which can be nested or overlap, and nesting
and overlapping are both needed for source code highlighing.[1][2]

I don't take a position on the "builtins are hostile to subclassing"
debate.  I can't recall ever noticing the problem, so I'll let you all
handle that. :-)


Footnotes: 
[1]  In Emacsen, tagged source text (overlays) is used not only for
syntax highlighting which presumably is nested (but TagSoup HTML!),
but also to implement things like hiding text, which is an operation
on raw text that need not respect any syntax.

[2]  XEmacs's implementation of syntax highlighting actually works in
terms of "extent fragments" which are non-overlapping, but they're
horrible to work with from a editor API standpoint.  They're used only
in the implementation of the GUI display, for performance reasons, and
each one typically contains a plethora of tags.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UVQ6PGUKF5EG6UZWOBI76ZQANNFVC5TS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Idea: Tagged strings in python

[Stephen J. Turnbull]

e...@emilstenstrom.se writes:

 > Seems simple enough, right? The problem is: There's no syntax
 > highlighting in my code editor for the three other languages.

Then you're not using Emacs's mmm-mode, which has been available for a
couple of decades.  Now, mmm-mode doesn't solve the whole problem --
it doesn't know anything about how the languages are tagged.  But this
isn't a problem for an Emacs shop, the team decides on a convention
(or recognizes a third party's convention), and somebody will code up
the 5-line function that font-lock (syntax highlighter in Emacs) uses
to dispatch to the appropriate the syntax highlighting mode.

AFAICS this requires either all editors become Emacs ;-) or all
editor maintainers get together and agree on the tags (this will need
to be extensible, there are a lot of languages out there, and some
editors will want to distinguish languages by version to flag syntax
invalid in older versions).  Is this really going to happen?  Just for
Python?  When the traditional solution of separating different
languages into different files is almost always acceptable?

There are other uses proposed for tagged strings.  In combination,
perhaps this feature is worthwhile.  But I think that on its own the
multiple language highlighting application is pretty dubious given the
limited benefit vs. the amount of complexity it will introduce not
only in Python, but in editors as well.

 > This makes for a horrible developer experience, where you
 > constantly have to hunt for characters inside of strings.

If this were a feature anyway, it would be very useful in certain
situations (for example dynamic web pages), no question about it.  But
mixed-language files are not something I want to see in projects I
work on -- and remember, I use Emacs, I have mmm-mode already.

 > If I instead use separate files, I get syntax highlighting and
 > auto-completion for each file, because editors set language based
 > on file type.

This is problematic for your case.  This means that the editor needs
to change how it dispatches to syntax highlighting.  Emacs, no
problem, it already dispatches highlighting based on tagged regions of
text.  But are other editors going to *change* to do that?

 > But should I really have to choose?

Most of the time, I'd say "yes", and you should choose multiple
files. ;-)  YMMV of course, but I really appreciate the separation of
concerns that is provided by separate files for Python code, HTML
templates, and (S)CSS presentation.

 > *Do we need a python language solution to this?*
 > Could the code editors fix this? There's a long issue thread for
 > vscode where this is discussed:
 > https://github.com/Microsoft/vscode/issues/1751 - The reasoning
 > (reasonable imho) is that this is not something that can be done
 > generally, but that it needs to be handled at the python vscode
 > extension level. Makes sense.

Makes sense, yes -- that's how Emacs does it, but Emacs is *already*
fundamentally designed on a model of implicitly tagged text.  Parsing
strings is already relatively hard because the begin marker is the
same as the end marker.  Now you need to tie it to the syntax
highlighting mode, which may change over large regions of text every
time you insert or delete a quotation mark or comment delimiter.  You
*can't* just hand it off to the Python highlighter, *every* syntax
highlighter that might be used inside a Python string at least needs
to know how to hand control back to Python.  For one thing, they all
need to learn about all four of Python's string delimiters.

And it gets worse.  I wonder how you end up with CSS and HTML inside
Python strings?  Yup, the CSS is inside a 

[Python-ideas] Re: Enhancing variable scope control

[Stephen J. Turnbull]

Anony Mous writes:

 > You want
 > You want
 > You want

You can't always get what you want
But if you try some time
You might find
You get what you need.
Batteries included![tm]

Jagger and Richards evidently knew a lot about language design. ;-)

 > It works very well, does exactly what I wanted, and improved my
 > particular circumstance significantly.

Nobody is asking you to give it up.  We're saying that based on our
limited experience, and Ruby's much greater experience, we don't want
it as a standard feature in the publicly shared Python language
implemented by this project.  Many of us program large systems in
Python, and we've found that certain restrictive rules that can be a
drag on individual programmers make it easier to work together on
them.

One of them is "no monkey patching", and it is accepted by consensus.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WSBT3PNKU5WKBPOAESVGGHNXVVANLKVS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Idea: PEP 3132 – Extended Iterable Unpacking for custom classes

[Stephen J. Turnbull]

Randolf Scholz writes:

 > Basically, I think it would be need if we could write
 > 
 > first, *tail = np.array([1,2,3])
 > 
 > and have tail be a np.ndarray.

 > Currently, the result is list.

I agree it might be nice, if we were designing the language from
scratch.  But it's *always* a list, and there are worse (more
incompatible) examples than ndarray.  Try it on a dict.  I guess the
current definition basically just converts "first, *tail = iterable"
to

first, tail = next(_ := iter(iterable)), list(_)

which is easy to understand and implement efficiently.

However implemented, this would undoubtedly break some existing code
that *needs* a list there, and make more existing code that already
converts the tail to some other type less efficient (although I do not
have an example where this would matter).

 > python could try initializing the object using the received type,

This would be backward-incompatible for builtin types, and obnoxious
if for some reason it fails (how do you propose this work for dict,
for example, which has non-mapping semantics as an iterable?)

 > or one could introduce a new dunder  classmethod __from_iterable__
 > that custom classes can implement.

Are there sufficiently interesting use cases, though?  The "first,
*tail" idiom is just syntactic sugar, but it does correspond to some
common situations that it expresses very neatly, such as function
argument lists where the first few arguments are mandatory and any
tail may be omitted.  Adding the simple implementation above is an
appropriate burden on both implementers and people who have to read it
for the benefit of this nice expression, and it doesn't really matter
which sequence is used in such cases -- the sequence is just going to
sit there waiting for a parser or an introspector to come along.

Making it more complex in any of the ways you propose imposes a fair
amount of cognitive cost on people reading code, because it
necessarily involves incompatibilities.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CRA2P4JXP3ZPOXNK7XV7B5OVTKLP3G7S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better (?) PRNG

[Stephen J. Turnbull]

James Johnson writes:

 > The scholars here are referencing white papers and using hardware and
 > software interchangeably (?) I am not a randomization expert, and I do not
 > seek academic recognition. It's a novel hack is all,

I'm no expert either, but hash functions and PRNGs have been around a
long time.  I'm sure this approach has been discovered independently.

Besides the Mersenne twister that Python uses, POSIX systems like
Linux, MacOS, and BSD provide a number of random number generation
facilities, briefly documented in the man pages.  BSD man pages for
random and arc4random mention the Fortuna algorithm and the arc4
algorithm.  At least the latter is similar to yours in that in "mixes
in" additional entropy gradually as more random numbers are generated.

For sources of "entropy" (random bits) the man page on /dev/random
and /dev/urandom provide nice brief explanations.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WXVOYIUFKWT6VNYOMVITCOHIHHNAV264/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better (?) PRNG

[Stephen J. Turnbull]

James Johnson writes:

 > I want to be good natured about it, not combative. I tried whatever random
 > function I found in documentation online, and used it. Without analyzing
 > the world, I was simply dissatisfied with the results.

On the Python lists, you'll get much better discussion if you're
specific about your "dissatisfaction".  (If it's just the repetition
problem, that's better handled by composing with a filter than by
altering the PRNG itself, which should be as close to uniform as
possible to make generating other distributions as simple as possible.)

Your method *as described* is unclear.  Don Knuth tried something
similar (you have better tools available than he did, but the basic
idea seems to be similarly "let's compose a few pseudo-randomizing
transformations"), and got an embarrassing result.  (Spoiler in
footnote [1] if you don't feel like digging through Seminumerical
Algorithms.)  You may have had more method than Knuth did (cf Dave
Mertz's statement that it "looks like Random123"), but again, it would
really help communication if you described why you think your PRNG
will do better than Python's current one.  It's a heavily researched
field for many decades now, and there are quite a few core devs with a
lot of knowledge and experience in the field (Python apps provide
quite of bit of the "attack surface" exposed on the Internet, it would
be irresponsible if there weren't!)

By the way, nobody here is going to laugh if it turns out that your
thinking was naive.  My point is to communicate more effectively, not
that your idea is bad (despite having a copy of Seminumerical
Algorithms, I'm not competent to evaluate your algorithm itself :-).
How you feel about "getting schooled" is a personal thing, but my
personal experience has been that there's an excellent education to be
had by posting naive ideas to this list. :-)

Sincere regards,

N.B. Deliberately not trimming so you have to work to see the
spoiler. :-)

 > 
 > Here’s what I settled on. I actually “put my thumb on the scales,” to rule
 > out repetitions for seven questions, so randomness didn’t ultimately answer
 > my question.
 > 
 > I’m not sure that the function I “got” was the Mersenne Twister. I do know
 > Mersenne was a truly great mathematician.
 > 
 > I acknowledge that updating with time.asctime() EVERY time I update the
 > hash would be random; I’m not qualified to know if it would be “more”
 > random. After you ask about the bell curve v square distributions, I am
 > quite out of my depth.
 > 
 > Here’s my “wizard” for anyone who wants to sell him cryptocurrency or
 > overcome his objections to the environmentalist agenda, or query him about
 > foreign policy. I hope you find it better than average.
 > 
 > https://drive.google.com/file/d/1EqQsMfBHDrNpOBQrI7CJxrpbowvKPD2G/
 > 
 > Regards,
 > 
 > James J
 > 
 > On Tue, Nov 15, 2022 at 4:31 AM Wes Turner  wrote:
 > 
 > > While FWIU it is advisable to keep seeding an RNG after startup - that's
 > > what e.g. rngd does - the cryptography.io docs do advise to just use
 > > `os.urandom()` (which is not the same as random.SystemRandom()?).
 > >
 > > There probably should be better default random in CPython; though I'm
 > > personally not at all qualified to assess, TIL about NIST 800-22 and FIPS
 > > 140-2 for evaluating sources of entropy.
 > >
 > > Differential entropy > Differential entropies for various distributions
 > > https://en.wikipedia.org/wiki/Differential_entropy
 > >
 > >
 > > ***
 > >
 > > (TIL that quantum information is never destroyed; so which physical
 > > processes are actually nonreversible like hashing and RNG are supposed to
 > > be is up for consideration as classical information is a subset of quantum
 > > information. Do black holes shift gamma radiation may actually be relevant!
 > > Perhaps a permanent magnet under a (double-jointed?) bar and ball pendulum
 > > on a magnetic bearing would produce enough Brownian motion to get enough
 > > uniform random to call it sufficiently entropic for purposes of CSPRNG?
 > > Nondeterministic NDE fluid calculations diffract into many possible
 > > outcomes, but a brute force combinatorial search of all the possible return
 > > values is still deterministically ordered. And the quantum computing folks
 > > are working on increasing coherence / reducing error propagation; like 
 > > ECC.)
 > >
 > > - [ ] DOC: The docs could advise regarding which decent enough open source
 > > hw RNG would be supported by os.urandom or random.SystemRandom if rngd is
 > > not configured to keep nondeterministically seeding with entropy that
 > > should presumably be from a Uniform random entropic process
 > >
 > > There should be better software random in python.
 > >
 > >
 > > On Tue, Nov 15, 2022, 1:19 AM James Johnson  wrote:
 > >
 > >> Thank you for replying with such specific assistance. I am made acutely
 > >> aware that I am only a Python enthusiast, and not an academic.
 > >>
 > >> Hashes are deterministic, not 

[Python-ideas] Re: Add mechanism to check if a path is a junction (for Windows)

[Stephen J. Turnbull]

Paul Moore writes:

 > While I frequently advocate on the side of "not every 3-line function needs
 > to be in the stdlib", there are a lot of convenience functions for Unix in
 > the stdlib

IMO "is_*" functions aren't exactly "convenience" functions, even if
they're only a couple of lines implemented in terms of stat.  I think
of them as "discoverability" functions -- by pulling these
distinctions out of class stat's data and documenting them as top-
level functions it's a lot easier to learn about and make distinctions
when you happen to coding in the neighborhood of "things you can do
with file system objects of type X but not quite with those of type X'
(which is very similar...)".

I mean, I'm not willing to die on the hill of that particular
terminology, but I do think it's worth making a distinction between
saving a few dozen keystrokes with a function that "would be obvious
to any skilled practitioner how to write it in 3 lines", and a
function that would require many a skilled practitioner a google and
reading a couple man pages to write in 3 lines.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3RGNBK6AGXUGWYTXD2KM2QYWNHSBAJZA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Is it possible to provide an official type annotation checker？

[Stephen J. Turnbull]

Christopher Barker writes:

 > Also: Static type checking is optional in Python. There are those of us
 > that are not convinced that static type checking is or should be a Python
 > best practice.

Why would you be?  Nobody (sane senior core) is advocating annotations
as a Python-wide best practice.  Whether annotations should be used or
not is a project question.  There are plenty of Python applications
where it would be insane to use annotations.  But there are others
where it's a good idea, even a best practice.

 > An official type checker would be an endorsement not just of that
 > particular approach to type checking, but also the concept itself —
 > I don’t think the community is ready for that.

I wish you'd stop fighting this battle.  It's over.  The type
annotations concept as a gradual and optional approach to type
checking is a done deal -- that check has been written, deposited,
*endorsed*, returned to writer, and locked away for the inevitable tax
audit by now.

If there are specific aspects of the annotations API that interfere
with existing practice or changes to the compiler or interpreter that
seem to require a type annotation is some context, that's another
matter.  I'm not saying that I think that those should always be
resolved in favor of alternatives to the type checking application --
I don't.  But it's certainly a valid issue and the alternatives have
every right to advocate for themselves.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VU2T3JQK5OH53QB6TMHTSDEPDLIBWHJM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Variadic patterns

[Stephen J. Turnbull]

Christopher Barker writes:

 > I don’t- I have the idea that Python itself is not really about
 > Types.

Of course it is, and always has been.  Types are even first-class
objects in Python.  What Python has never been about is universal
type-checking.

 > Honestly, it makes me nervous when supposedly “optional” type hints start
 > making their way into built in parts of the language and standard idioms.

Type hints are easily recognizable from the syntax, and match
statements aren't that.  This isn't creeping typing-ism at all.

Python has *always* (at least since 1996 or so when I first
encountered it) made "look before you leap" and "dispatch on type"
styles possible.  The match statement is clearly about enabling
exactly the latter kind of programming:

 > Matt del Valle writes:

 > > Matching on type is one of the main examples the tutorial PEP showed off
 > > (see the example where an event is matched against several different types
 > > such as Click, KeyPress, or Quit).
 > 
 > I guess my feeling is that there are better ways to solve those kinds of
 > problems in Python.

Like what?  Dispatching on type is fundamental to event-oriented
programming.  Of course you can write

if isinstance(event, Click):
x = event.position.x
y = event.position.y
button = event.b
handle_click(x, y, button)

or

if isinstance(event, Click):
handle_click(event.position.x, event.position.y, event.button)

instead of

match(event):
Click(position=(x, y), button):
handle_click(x, y, button)

and I can see an argument that the former is "just as good" as the
latter and we don't need match, but now that we do have the match
statement, I can't see any argument that either if form is *better*.

 > This is the key point — and you are quite right. I’ll let others
 > comment on whether this extension to pattern matching makes sense —
 > I haven’t really dig into it enough to have an opinion.

About Matt's variadic container matching:  My guess is that it was
considered, but it's not easy to find pleasant applications because
you can only do (x, y, *rest) = whatever destructuring on the whole
sequence, which is a one-liner in a case that catches sequences.  The
typecheck for simple types such as str or even a user-defined class is
also a one-liner (all(isinstance(o, type) for o in container).  I
don't think turning those three lines into one is irresistably
attractive, which means it's most interesting for a case where you
have a "type" defined inline (typically as a dict, I guess).  I'm not
sure how common that is.  And in a case like:

match mailbox:
case [*{"From" : author, "To" : recipient, "_payload" : body}]:
do_something(mailbox)

isn't do_something very likely to be of the form

for message in mailbox:
do_something(message)

and you have to destructure message explicitly anyway?  So in such
cases I would expect that

for message in mailbox:
match message:
case {"From" : author, "To" : recipient, "_payload" : body}:
do_something(author, recipient, body)

is a simple and obvious transform.

On the other hand, typecheck failures on objects in a container can be
expensive if the container is large.  You can only recover from them
by checking for a container of untyped objects, and then handling that
elementwise anyway.

So on general principles, I don't think this idea is bad on the face
of it, but it's not obviously great, either.  The usual policy in such
cases is to wait for a killer app before adding the feature (cf. the @
operator, which waited for literally decades despite having an obvious
killer app, at least for a couple of communities that were pretty
large then and are huge now).  If you really could use match(data) to
parse data and create a DataFrame in pure Python, that would be a
strong PoC, though not yet a killer app since I doubt Pandas would use
it.  Obviously something sufficiently performant to be widely used
would be nice.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R2LI27QIV4GDM4PTBI5U63DDPPOMAB6T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Feedback before submission of PEP 661: Sentinel Values

[Stephen J. Turnbull]

Christopher Barker writes:

 > > For example, a common confusion is to identify None with NULL in
 > > DB, ... because most frameworks do not distinguish "Do not
 > > filter" and "Keep only rows where value is NULL",
 > 
 > Good example— wouldn’t it be nice if all the database interaction packages
 > used the same Sentinel for this? In that case it would go in the DB api,
 > but the idea holds.

The question is whether it's an *example* of a common need, or a
*nearly unique use case* of a generally useful distinction that
justifies a new (semi-)singleton.  Tal says that at least in the
stdlib and the other cases he's seen it's almost always an
application-specific need.  We need to see a *list* of generally
useful sentinel classes that aren't served by Tal's "_local_sentinel =
Sentinel()" idiom.  Otherwise such sentinels can be added ad hoc to
specific (though widely used) modules as module-specific APIs.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZDRV5JGZSPIESIOUNWPSNAO2O3TU5FNT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Feedback before submission of PEP 661: Sentinel Values

[Stephen J. Turnbull]

Christopher Barker writes:

 > > The current design is that sentinels with the same name from the same
 > > module will always be identical. So for example `Sentinel("name") is
 > > Sentinel("name")` will be true.
 > 
 > Hmm -- so it's a semi-singleton -- i.e. behaves like a singlton, but only
 > in a particular module namespace.

I don't see why this is a problem.  Sentinels are "out of band"
signals, they're invalid data *for the problem at hand*.  That doesn't
mean they're *always* invalid data.  For example, Unicode surrogate
characters are invalid in conforming streams of characters like Python
str, but Python uses them internally anyway as "sentinels" meaning
"byte value #XX encountered in input, but is invalid there".  (You
might not agree that these are sentinels, but I'm hard put to see why
my interpretation is incorrect.)  And, of course, Unicode itself uses
them in concrete representations of characters, specifically UTF-16.

So sentinel usage is fundamentally private to an application as far as
I can see, and the sentinel value is fundamentally arbitrary, as long
as it's not valid data for the application.

 > I would like to see a handful of common sentinels defined. Is there a plan
 > to put a few that are useful for the stdlib in the `sentinel`
 > module?

I don't think this works.  You end up with the problem that None has
as a sentinel: mostly it's great, but occasionally application's
interpretation gets in the way of the sentinel semantics because in
the end Python considers it a valid datum.  For modules to be
composable, they need to agree on the *exact* semantics of the
sentinel.  If a convention based on one of the true singletons (None,
Ellipsis, True, False) doesn't work, you probably want a private
sentinel anyway.

IMO YMMV

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FR6MA4ZR2UY5BTLM23BVNHGKSEGFBFQW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add InvalidStateError to the standard exception hierarchy

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > A closed file is not *invalid*, it is just closed.

I think it's not very useful to focus on the individual words used
here.  I believe that Steve J's idea here is related to the fact that
an object is a namespace and an environment.  So the operation is not
"read from specified file", it's "read as appropriate in the specified
environment" (duck-typing), so the state of the environment of the
call (which is the implicit argument "self") is invalid.

I myself would argue it's not that the environment is in an invalid
state for reading, but that reading is an invalid operation on a
closed file.  I note that XEmacs Lisp's docstring for the error
function says:

`invalid-operation' refers to all cases where code is trying to do
something that's disallowed, or when an error occurred during an
operation. (These two concepts are merged because there's no clear
distinction between them.)

I want to emphasize the remark in parentheses.  It's frequently *very*
difficult to make distinctions that will be clear to all users of the
API, or even to all API users named "Steve". :-)  In my personal
opinion, the XEmacs Lisp hierarchy does a good job of specifying a
hierarchy that lends itself to clear distinctions.  YMMV on whether
that hierarchy is a good one, but I think the point is important:
there are no absolutes here, rather the quality of a distinction is
relative to the rest of the hierarchy, it depends on how well it fits
in.

Bottom line: let's focus the argument on (1) where in the Python
exception hierarchy the new builtin exception would go, and (2) per
Paul Moore, if it could be used in the interpreter (I will add, or in
a variety of stdlib modules), and where.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4L6WJ2HC6EN4YNIFORIVIRDUARUQG4XT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add InvalidStateError to the standard exception hierarchy

[Stephen J. Turnbull]

Steve Jorgensen writes:
 > Paul Moore wrote:

 > > The built in exceptions are ones that are raised by the core interpreter.

 > OK, but by that logic, why do we have standard exceptions like
 > `ValueError` when we could define custom exceptions for the cases
 > where that should be raised?

I get 154 hits for PyExc_ValueError in cpython/Python.  That's exactly
the logic that Paul mentions. :-)

I suspect that the reason that there is no PyExc_InvalidStateError is
that the interpreter rarely can detect it.  For example, in the
following very contrived example

def getcontents(file):# argument should be "filename"
# Oops, forgot to open the file named by `file`.
return file.read()

you'll get an AttributeError.  If you typecheck `file` at the top of
the function, ValueError is appropriate.  It's not hard to contrive a
case where you can detect it:

>>> with open(RandomExistingFile) as f:
...  f.close()
...  f.read()
... 
Traceback (most recent call last):
  File "", line 3, in 
ValueError: I/O operation on closed file.

and Python chooses to shoehorn the round InvalidStateError peg into
the square ValueError hole.  This isn't entirely implausible in
Python's implementation of object-oriented programming: the object `f`
is an invalid value for the `self` argument of the read function.

So, assuming you use this "it's in the interpreter" logic in
advocating this new Error, I would say collect examples of other
errors raised where you would prefer InvalidStateError.  If they're
consistently ValueError, I would say you have a case.  Probably you
need to derive InvalidStateError from ValueError for backward
compatibility.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MZQQVUWFB5P35EBYPL2TJRUE5OCQ7XB3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] On "batteries included" [was: standard library Multiset ...]

[Stephen J. Turnbull]

Christopher Barker writes:

 > It seems that’s just what you want. Honestly, despite the idea of
 > “batteries included”, it is very common these days to need a third party
 > lib.

Just because your Gameboy came with batteries, doesn't mean that your
favorite game was included.

The idea of "batteries included" is that everything you need to get
started on "real work" is included, not that your "real work" is
already done for you.  It also needs to be one size fits all (does
anybody else remember Mad Magazine's sexist "one size fits all"
cartoon? -- sometimes it "fits" but it's been stretched beyond all
reason!)  And its dev cycle needs to fit the stdlib's.  A lot of
packages of quite general interest will be releasing feature releases
quarterly, or even monthly.

Of course the lines to draw for all of those desiderata are judgment
calls, but the principles are valid.

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YFOUNQ4TKBJUGYHCOL3RQDOLVTZ5334E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] [Python-ideas][RESEND] PEP Idea: native f-string support as a match pattern

[Stephen J. Turnbull]

Sorry, I accidentally sent before I was done.

Tushar Sadhwani writes:

 > Since Python has built-in syntax for interpolated strings, I
 > believe it's a good area to idea to extend it to pattern matching,
 > like so:
 > 
 > def unquote(string: str) -> str:
 > match string:
 > case f'"{value}"':
 > return value
 > case f"'{value}'":
 > return value
 > case _:
 > return string

I think this is a pretty unconvincing example.  While people seem to
love to hate on regular expressions, it's hard to see how that beats

def unquote(string: str) -> str:
m = re.match(r"""^(?:
 "(.*)"# "-delimiters
|'(.*)'# '-delimiters
|(.*)) # no delimiters
$""",
 string,
 re.VERBOSE)
return m.group(1) or m.group(2) or m.group(3)

and this is absolutely clearer than either pattern-matching approach:

def unquote(string: str) -> str:
# Gilding the lily, but it's obvious how to extend to other
# symmetric delimiters, and straightforward for asymmetric
# delimiters.
for quotechar in ("'", '"'):
if string.startswith(quotechar) and string.endswith(quotechar):
return string[1:-1]
else:
return string

 > Doing this with current match syntax is not as easy.

Sure, but that's why we have .startswith and .endswith, and for more
complex cases, why we have regular expressions.  Chris Angelico has
proposed adding (something like) C's scanf to the stdlib, as well.

 > I have other reasons to consider this idea as well,

Given the above, I really think you need to bring those up if you want
to pursue this idea.

 > but before I try to pursue this, I'd like to know if something like this was
 > already discussed when the proposal was being made?

Looking at PEPs 634-6, and especially 635, I doubt it (but I didn't
participate in those discussions).  On the one hand, the match
statement is for matching (a subset of) Python expression structure,
not string structure.  On the other, although an f-string *is* an
expression, it doesn't really feel like one, to me, anyway.  Also, I
think it would be miserable to code (do you really want to support the
full replacement field syntax with widths, justification, precision,
and more) and the bikeshedding will be horrific (supposing you support
a restricted field syntax with no width and precision, is

case f"{value:03.2f}":

an error or do you ignore the padding, width, and precision?)

It's a very interesting suggestion, but I suspect the bikeshedding is
going to be more fun than the implementation.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H5KDLU7HJO7ZNUYOLQ5QK6I3NCNGKKUM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP Idea: native f-string support as a match pattern

[Stephen J. Turnbull]

Tushar Sadhwani writes:

 > Since Python has built-in syntax for interpolated strings, I
 > believe it's a good area to idea to extend it to pattern matching,
 > like so:
 > 
 > def unquote(string: str) -> str:
 > match string:
 > case f'"{value}"':
 > return value
 > case f"'{value}'":
 > return value
 > case _:
 > return string

I think this is a pretty unconvincing example.  While people seem to
love to hate on regular expressions, it's hard to see how that beats

def unquote(string: str) -> str:
m = re.match(r"^(?:\"(.*)\"|'(.*)'|(?Pvalue3))$", string)

 > 
 > Doing this with current match syntax is not as easy.
 > 
 > I have other reasons to consider this idea as well, but before I try to 
 > pursue this, I'd like to know if something like this was already discussed 
 > when the proposal was being made?
 > ___
 > Python-ideas mailing list -- python-ideas@python.org
 > To unsubscribe send an email to python-ideas-le...@python.org
 > https://mail.python.org/mailman3/lists/python-ideas.python.org/
 > Message archived at 
 > https://mail.python.org/archives/list/python-ideas@python.org/message/4D2CRUSACUYK2VNYHULBUQ525QAF7XDV/
 > Code of Conduct: http://python.org/psf/codeofconduct/
 > 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PCHVUOR6MZYWSAVYUVZ4E3ZCEH73AFZF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Void type

[Stephen J. Turnbull]

Mathew Elman writes:

 > To answer how this _could_ work, Undefined would be a new NoneType

My example is intended to refer specifically to the alternative
semantics where 'undefined' is not allowed outside of function
prototypes (or other specified contexts, for that matter).  The point
of the example is that it will be possible to capture that singleton
and use it anywhere by a different name.  And you just know that
somebody will find a reason to do it.

 > I am not sure I understand your example
 > > notdefined = None
 > > def foo(x=undefined):# here
 > > notdefined = x

You're right, this should be "global notdefined; notdefined = x".

 > > return wrapped_function(x)   # there
 > > foo()
 > > print(notdefined)

What does that print?  Or does it error?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/37546QCEKSRTDZR2GODKKX4JFQDGQ7B4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Void type

[Stephen J. Turnbull]

Mathew Elman writes:

 > I believe this is a rebirth of a request that has come up many
 > times before, which is to have something like javascript's
 > `undefined` where it means "use the default value" if passed to a
 > function that has a default value or "value not provided" (slightly
 > different to "None").
[...]
 > "undefined" would either have to be an alias for "None" everywhere
 > _except_ function signatures, or be only allowed in function
 > signatures and have special handling.

Either would be a significant complication in Python semantics,
though.  And I don't really see how the latter works:

notdefined = None
def foo(x=undefined):# here
notdefined = x
return wrapped_function(x)   # there
foo()
print(notdefined)

To get this "use the default" functionality from #here to #there,
either the above has to allow you to capture the object representing
undefined but you're not allowed to use the name "undefined" for it
(very un-Pythonic), or there has to be some horrifying magic, maybe
macro-like, that turns the name "x" into a name that isn't there.

 > It opens up a can of worms in javascript but has its uses.

If you're phishing, I guess worms are useful.  *shudder*

Wisecracks aside, I'm not sure I want to read code designed so that
'undefined' is genuinely useful.  I'd need to see real code that uses
it, plus the translation to Python, plus some argument why it has to
be that way, before I'd be willing to go higher than -1 on this
complication.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QOX623SJC65TB5U4TIZRC7EW6T2K2XT2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Confusing naming of Optional type should be changed

[Stephen J. Turnbull]

Kevin Mills writes:

 > While confusion with optional arguments is somewhat unfortunate,
 > the name came from an already established convention. A lot of
 > languages have exactly the same concept, varying between names like
 > Optional, Option, and Maybe. I don't think coming up with a
 > Python-specific name for the same thing is necessarily better.

This seems likely to be more or less moot.  If you're interested, see
Guido's (!!) new PEP 695 (pretty sure that's the number, need to go to
meeting that is starting, sorry).

 > 
 > Of course, it's immaterial now that the union syntax is preferred.
 > ___
 > Python-ideas mailing list -- python-ideas@python.org
 > To unsubscribe send an email to python-ideas-le...@python.org
 > https://mail.python.org/mailman3/lists/python-ideas.python.org/
 > Message archived at 
 > https://mail.python.org/archives/list/python-ideas@python.org/message/4IAHIQ4V6UOGH2AJX46KWTXVBUJEP3MJ/
 > Code of Conduct: http://python.org/psf/codeofconduct/
 > 
 > 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FDTTV5Q7LSN3MAJGAUWPQ64Z4HULS7LX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Confusing naming of Optional type should be changed

[Stephen J. Turnbull]

Jelle Zijlstra writes:

 > In fact, `typing.Optional` means that something can be None, *not*
 > that it is an optional argument.

You're missing my point.  Yes, the *implementation* is that the object
can be None.  But that's far more clearly expressed by Union[T, None]
if that's what you *mean*, and it only costs 3 characters.  Optional
clearly was intended to mean that this type is for use in optional
arguments.

Having "Optional" at all may have been a bad choice, since (1) it is
redundant given you still need to specify "= None" to actually make it
optional, and (2) usually shouldn't be used for optional arguments
that don't default to None (because in such cases None usually isn't
an acceptable argument).  I am persuaded of that now: it doesn't
express the idea "this type is for use in optional formal arguments".

But I am convinced by arguments about the *language* (including type
expressions) that are about *what it expresses*, as in the paragraph
above (natural language) or your example function f (code example).
Arguments about what something *is* in the implementation, or vague
statements that things "could" be misinterpreted aren't terribly
helpful to me, YMMV.  And I think they lead to bad decisions about the
language.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LLHRUM3CNBLVAX7VCFTZ6ZUPBVEXNBS7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Confusing naming of Optional type should be changed

[Stephen J. Turnbull]

nveric...@gmail.com writes:

 > I accidentally created another thread in python-dev as I mentioned
 > above, but ideally Optional and Union should both be deprecated and
 > phased out for the new syntax.

I think a formal deprecation is a bad idea.  An annotation is an
object, which has a type.  Both the object and the type are going to
be visible to users.  We don't want there to be warnings about them
forever, and Python as a rule does not formally deprecate working code
that is expected to continue to work indefinitely.

I suspect that it would be difficult to get a stylistic deprecation
into PEP 8 (IIRC type annotations are still not allowed in stdlib code
so it would be considered out of scope), but you could try lobbying
the maintainers of linters.

BTW, I disagree with your arguments that Optional and Union are
misleading names that can be easily misunderstood, especially in the
usual context of formal arguments in function definitions.  The
suggestion of "Noneable" takes the Pythonic implementation of optional
arguments (by defaulting to None) too seriously, at the expense of the
syntactic intention: an argument that may be omitted.  Among other
things, very frequently 'None' is *not* an allowed value in the body
of the function, some other value is immediately substituted (and PEP
671 explicitly tries to automate this process for the common case of a
mutable default that should be constructed at call time, so that even
the idiomatic "if arg is None" statement is left out).

"Optional" is the normal term used for such arguments, "union" is the
technical term for types composed of being either this or that type.
If you need to know any more than that about those types, you're going
to have to study up no matter what terms are used.  That's just the
nature of using natural language words to name things that have a
precise definition and implementation in software.  Study is required.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LELXP5FPJZWF36NR423ZLVOHGXQTUVKL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Stephen J. Turnbull]

Chris Angelico writes:

 > So the only way around it would be to make the defer keyword somehow
 > magical when used in a function signature, which kinda defeats the
 > whole point about being able to reuse another mechanic to achieve
 > this.

The defer keyword is already magical.  Overloading it with more magic
doesn't bother me.  The question of internal consistency of the
various magics does bother me.

 > Also, it would create some other oddity, depending on which way
 > this is handled:
 > 
 > _default = defer []
 > def foo(cookiejar=_default):
 > 
 > Does this also get the magic, or doesn't it? Either way, there'd be a
 > really weird inconsistency here.

Don't know, need to think about the definition and implementation of
the magic first.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SZRYDYQDDGTWPFALR3GRKFGDUMTXC5RZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Stephen J. Turnbull]

David Mertz, Ph.D. writes:
>>>>> On Fri, Jun 24, 2022 at 3:50 AM Stephen J. Turnbull 
>>>>>  wrote:

 > > I'm suggesting modified semantics where deferreds can be a proxy
 > > object, whose normal reaction to *any* operation (possibly
 > > excepting name binding) is
 > >
 > > 1.  check for a memoized value,
 > > if not found evaluate its stored code, and memoize the value
 > > 2.  perform the action on the memoized value
 > 
 > I think I like these semantics better than those my draft proposal.  I
 > haven't had a chance to enhance the proto-PEP more in the last few days
 > (other work).  But all of these comments are extremely helpful, and I'll
 > have a better version in a few days.  Hopefully I can address many of the
 > concerns raised.

We (not sure how much help I'll be, but I'm in) need to deal with
Chris A's point that a pure memoizing object doesn't help with the
mutable defaults problem.  That is with

def foo(cookiejar=defer []):

foo() produces a late bound empty list that will be used again the
next time foo() is invoked.

Now, we could modify the defer syntax in function parameter default
values to produce a deferred deferred object (or, more likely, a
deferred object that lacks the memoization functionality).  But I
suspect Chris will respond with (a polite expression with the
semantics of) the puke emoji, and I'm not sure I disagree, yet. ;-)

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IBUB7GMZMRPXDAYNRNT7A5CAPEKE3RO3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Stephen J. Turnbull]

Barry writes:
 > > On 23 Jun 2022, at 08:27, Stephen J. Turnbull  
 > > wrote:

 > Interest idea that ref does not auto evaluate in all cases.
 > I was wondering about what the compile/runtime can do it avoid the costs
 > of checking for an evaluation.

I think the main thing to do is to put the burden on the deferred
object, since it has to be something special in any case.

 > > Now consider a = b + 0.  b.__add__ will be invoked in the usual way.
 > > Only if b is a deferred will evaluation take place.
 > 
 > But the act of checking if b is deferred is a cost I am concerned about.

That's true in David's proposed semantics, where the runtime does that
check.  I'm suggesting modified semantics where deferreds can be a
proxy object, whose normal reaction to *any* operation (possibly
excepting name binding) is

1.  check for a memoized value,
if not found evaluate its stored code, and memoize the value
2.  perform the action on the memoized value

That means that in the statement "a = b + 0", if b is an int,
int.__add__(b, 0) gets called with no burden to code that uses no
deferreds.

Then the question is, why do we need syntax?  Well, there's the PEP
671 rationales for deferring function argument defaults.  There is
also the question of whether name binding should trigger evalution.
If so,

a = defer b

would (somewhat similar to iter) defer normal b (this could be
optimized by checking for non-mutables) and "re-defer" a deferred b
(ie, just bind it to a without evaluation).  The same consideration
would apply to "as" and function arguments (possibly with different
resolutions!  I'm probably missing some name-binding syntax here, but
it should be clear where this going).

 > I would think that it’s not that hard to add the expected check
 > into the python ceval.c
 > And benchmark the impact of the checks. This would not need a full
 > implementation of the deferred mechanism.

+1  Although I'm not going to spend next weekend it. ;-)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QL6F7UHQKBM3NKFCQ3O525EUZAGYVCN7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Stephen J. Turnbull]

Barry Scott writes:

 > I can think of ways to implement evaluation-on-reference, but they
 > all have the effect of making python slower.

Probably.

 > The simple
 > 
 >  a = b
 > 
 > will need to slow down so that the object in b can checked to see
 > if it need evaluating.

No, it doesn't.  Binding a name is special in many ways, why not this
one too?  Or "a = a" could be the idiom for "resolve a deferred now",
which would require the check for __evaluate_me_now__ as you say.  But
such simple "a = b" assignments are not so common that they would be a
major slowdown.  I would think the real problem would be the "oops" of
doing "a = b" and evaluating a deferred you don't want to evaluate.
But this isn't a completely new problem, it's similar to a = b = []
and expecting a is not b.

Now consider a = b + 0.  b.__add__ will be invoked in the usual way.
Only if b is a deferred will evaluation take place.

So I don't really see the rest of Python slowing down much.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XYCOZYJSNDZTRMMBDNIL4E62SVIYHRBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Stephen J. Turnbull]

@Chris

My bottom line, as I wrote before, is that even if this were
introduced, I probably will continue to default to

def foo(arg=None):
if arg is None:
arg = default

in my own code until I start seeing "def foo(arg=>default)" in a lot
of code I read.  Since Mailman generally supports about 4 Python
versions, that means I won't see it in Mailman until 2027 or so.

But I'm not George Bush to say "Read my lips: no new (syn)taxes!"

Unless somebody comes up with some new really interesting use case, I
think the suggestion somebody (sorry to somebody!) made earlier to
"Just Do It" and submit to the SC is the right one.  Both David and I
are convinced that there is value-added in late binding for new
mutables and defaults that are computed from actual arguments, even if
we're not convinced it's enough.  The proposal has plenty of fans, who
*are* convinced and *will* use it.  I don't see a prospect for that
new really interesting use case, at least not here on Python-Ideas,
the discussion is just variations on the same themes.  On the other
hand, a PEP under consideration may get a little more interest from
the Python-Dev crowd, and obviously the SC itself.  They may have use
cases or other improvements to offer.

"Now is better than never."  The SC will let you know if the companion
koan is applicable. ;-)

@Chris You may or may not want to read my variations on the themes. ;-)

Chris Angelico writes:

 > > In the numeric stuff, if I have:
 > >
 > > newarray = (A @ B) | (C / D) + (E - F)
 > >
 > >
 > > That's @, |, /, +, and -.  So 5 operators, and 25 "complexity
 > > points".  If I added one more operator, 36 "complexity points"
 > > seems reasonable.  And if I removed one of those operators, 16
 > > "complexity points" feels about right.
 > 
 > For my part, I would say that it's quite the opposite. This is three
 > parenthesized tokens, each of which contains two things combined in a
 > particular way. That's six 'things' combined in particular ways.
 > Cognitive load is very close to this version:
 > 
 > newarray = (A * B) + (C * D) + (E * F)

I don't have the studies offhand, but "7 plus or minus 2" is famous
enough, google that and you'll find plenty.  I'll bet you even find
"cognitive complexity of mathematical formulae" in the education
literature.  (And if not, we should sue all the Departments of
Education in the world for fraud. ;-)

I do have the words: "this is a sum of binary products".  This
basically reduces the cognitive complexity to two concepts plus a scan
of the list of variables.  Given that they're actually in alphabetical
order, "first variable is A" is enough to reproduce the expression.

That's much simpler than trying to describe David's 5-operator case
with any degree of specificity.  Or even just try to reproduce his
formula without a lot of effort to memorize it!  Also, just from the
regularity of the form and its expression as an algebraic formula, I
can deduce that almost certainly A, C, and E have the same type, and
B, D, and F have the same type, and very likely those two types are
the same.  Not so for the five-operator case, where I would be
surprised if less than 3 types were involved.

Of course, this type information is probably redundant.  I probably
remember not only the types, but lots of other attributes of A
through F.  But this kind of redundancy is good!  It reinforces my
understanding of the expression and the program that surrounds it.

 > even though this uses a mere two operators. It's slightly more, but
 > not multiplicatively so. (The exact number of "complexity points" will
 > depend on what A through F represent, but the difference between "all
 > multiplying and adding" and "five distinct operators" is only about
 > three points.)

That may be true for you, but it's definitely not true for my
economics graduate students.

 > > Sure, knowing what `hi` defaults to *could be useful*.  I'm sure
 > > if I used that function I would often want to know... and also
 > > often just assume the default is "something sensible."  I just
 > > don't think that "could be useful" as a benefit is nearly as
 > > valuable as the cost of a new sigil and a new semantics adding to
 > > the cognitive load of Python.
 > 
 > Yes, but "something sensible" could be "len(stuff)", "len(stuff)-1",
 > or various other things. Knowing exactly which of those will tell you
 > exactly how to use the function.

@David:  I find the "hi=len(stuff)" along with the "lst=[]" examples
fairly persuasive (maybe moves me to +/- 0).

@Chris:  It would be a lot more persuasive if you had a plausible
explicit list of "various other things".  Even "len(stuff) - 1" is
kind of implausible, given Python's consistent 0-based indexing
and closed-open ranges (yeah, I know some people like to use the
largest value in the range rather than the least upper bound not
in the range, but I consider that bad style in Python, and they
denote the same semantics).  

[Python-ideas] Re: Bare wildcard in de-structuring to ignore remainder and stop iterating (restart)

[Stephen J. Turnbull]

Lucas Wiman writes:

 > That said, the * syntax feels intuitive in a way that / doesn’t.

I disagree.  In C-like languages, it says "dereference a pointer" (ie,
use the content at the pointer).  In Python, it's used for
destructuring iterables, ie, use the content at the iteration pointer
by packing or unpacking.  By contrast, "/" doesn't have a unary
meaning in any language I know of (well, Lisp, but in Lisp it's just a
symbol that happens to have a built-in function definition).

 > I’d suggest:
 > x, *… = foo
 > This seems unambiguous and fairly self-explanatory.

I advocated just "..." myself, so obviously I'm biased, but I don't see
what prepending "*" says that "..." by itself doesn't.

Steven d'Aprano wrote:

 > > I like "/" because it reminds me of the slash in "No Smoking" signs, and
 > > similar. As in "No (more) iteration".

And in Python its only non-binary use is to say "no more positional-
only parameters."  I like "..." better for its suggestion that there's
more to come ;-) but objectively I guess "/" is just as good. :-)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LXGNFHFZZXD7BHOWDVOOA5WYR7Q32TGO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a line_offsets() method to str

[Stephen J. Turnbull]

Jonathan Slenders writes:

 > Good catch! One correction here, I somewhat mixed up the benchmarks. I
 > forgot both projects of mine required support for universal line endings
 > exactly like splitlines() does this out of the box.

I can't remember ever seeing an application where such a method is not
only useful but needs to be extremely efficient, so I'm a little
suspicious of the generality of the need.

On the other hand, for this particular case you should be able to just
subtract the string allocations from .splitlines.  How about
generalizing it into a maplines method, whose basic internal function
is to generate an iterator over (line_start, line_end) pairs (line_end
is useful because the separator is not necessarily a single
character).  Then it could optionally take a function to map over
those, as well as an optional separator string.  I suppose it would
usually be a single character, except for universal newlines, which
would be the default case both because it's so common and because it's
not a constant string.

This would then be the implementation of splitlines, as well as a
mapping tool for records (usually lines, but could be paragraphs) in a
string.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R5DI64RFOJYQ57NHNMBKC3RPHZJ2KRPV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > The match...case statement didn't "need" keywords either, we could have 
 > picked symbols instead if we wanted to look like APL. Remember that 
 > keywords have advantages as well as disadvantages. Given the existence 
 > of community support for keywords, the PEP should make the case that 
 > symbols are better in this case.

The argument I would make is that in the presence of type annotations
a keyword adds a lot of verbosity, and is possibly confusable (ie,
ignorable) as a type component, and that that is sufficient argument
for a symbol rather than a keyword.

 > This raises another choice: should lazy defaults be evaluated before 
 > entering the body of the function, or at the point where the parameter 
 > is used? Which would be more useful?



Chris makes the point that we've been through this before so I won't
belabor that point.  Both are potentially useful.  Chris wants syntax
for the common pattern

def foo(arg_with_new_empty_list_default=None):
if arg_with_new_empty_list_default is None:
arg_with_new_empty_list_default = []
# do stuff

and variants (eg, where the default is a computation expensive enough
that you don't want to do it if the argument is never defaulted).  I
don't find that burdensome enough to want syntax, and I can't guess
how quickly I'd start using it if available, that probably depends on
how often the community seems to be using it (ie, in the code I'm
reading).

I can't really guess how useful the "use point" version would be.
It's not a pattern I've used, I use a zero-argument function very
occasionally but I can't recall a case where I used a lambda (lambdas
are kinda annoying).  Adding the parens at the call is easy to forget,
but not that huge a burden either.  The only use case I can think of
offhand is where the default is such an expensive computation that you
don't want it done unless its result is actually used, and it might
not be:

def foo(flag, arg=None, *otherargs):
if flag:
if arg is None:
arg = expensive_function
return arg(*other_args)
else:
return a_constant

On the one hand, it seems likely that a very expensive function is
expensive enough to avoid at the cost of an obscure default and an if
in the definition of foo.  On the other the whole scenario seems
rather contrived and not worth syntax.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6Z7GD6STJQP7ABKINIMBNCVBS4BOSLOG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Stephen J. Turnbull]

Chris Angelico writes:

 > And there we have it. People are complaining loudly, but then ALSO
 > saying that they don't support the proposal anyway. Why are you
 > bothering to debate this if you've already made your decision?

I can't speak for Brendan, but I have two reasons for discussing
despite being broadly in sympathy with David Mertz's reasons for
opposing the proposal:

1.  Often enough I've disagreed with a proposal, only to see it
implemented.  Of course, I want the least bad version from my
point of view.

2.  More altruistically, even though I disagree with a proposal, I
would be ashamed if I thought there was a way to improve a
proposal, and didn't mention it in the hope of sabotaging the
proposal's chances by leaving it less than it could be.

That doesn't mean in either case I'm right, of course, but at least I
can *try*. :-)

Regards,
Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P7K2HAH3UM2OOONOOV45FSB3EUNDDA4C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Stephen J. Turnbull]

Chris Angelico writes:

 > but at very least, if it isn't *obvious*, it should at least be
 > *unsurprising* if you then get UnboundLocalError.

+1 I think this (== "obvious in hindsight") is probably a better
criterion than "obvious" (from the definition) when an error *will* be
raised.  (That's as a general principle.  Ie, it doesn't need to be
obvious that you will *get* feedback from the system when you do
something, but it should be obvious what the feedback means when you
*do* get it.)

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CTTX7MXMYMZEMDGZ3WF7Q5IHMT5JEWWD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bare wildcard in de-structuring to ignore remainder and stop iterating (restart)

[Stephen J. Turnbull]

Steve Jorgensen writes:

 > That leads me to want to change the proposal to say that we give
 > the same meaning to "_" in ordinary destructuring that it has in
 > structural pattern matching,

This is already valid syntax with different semantics.  Given the
existence of islice, there's really no excuse for breaking this, if
this were the only way to implement your syntax I'd be a solid -1 (as
it is I'm a -0 perhaps maybe I don't know ;-).

I could imagine a token other than "*" being used for this to avoid
the potential confusion or typo between "a, * = x" and "a, *_ = x",
such as Ellipsis or even None.  (Both are currently "cannot assign to"
errors.)  The best list of use cases for ellipsis I've found in a
quick look is https://python.land/python-ellipsis, and I don't see any
gotchas for your syntax (with ellipsis instead of star) there, but I
also didn't look very hard, and didn't think about the walrus operator
which I guess is a potential gotcha generator.

Steve



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HDMLI4AXBPYQNRO4I5ZOJ7RYATUZQ774/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: A “big picture” question.

[Stephen J. Turnbull]

Steven D'Aprano writes:
 > On Wed, Jun 08, 2022 at 06:51:54AM -0500, James Johnson wrote:
 > 
 > > When an amateur develops code incorrectly, s/he sometimes ends up with a
 > > code object that doesn’t run because of intermediate compiler
 > > optimizations.
 > 
 > If that happens, that's a bug in the compiler. Optimizations should 
 > never change the meaning of code.

This isn't quite true.  Languages (mostly low-level) frequently leave
behavior of legal syntax partly undefined in order to allow efficient
implementation on various architectures, or because it's unclear how
it will be used in the future (function annotations, I be lookin' at
you!)

 > If you have an example of this, where the compiler optimization changes
 > the meaning of Python code beyond what is documented, please raise a bug 
 > report for it.

OK, you've mostly taken care of the letter of my comment.  But I still
think it's worth pointing out that the documentation is frequently
incomplete for various reasons.

 > But I doubt you will find any, because Python performs very, very few 
 > optimizations of the sort you are referring to.

True, although it might in the future.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QEHF3IDM24LVD6PZY7OF4YJFM3FU75T3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Null wildcard in de-structuring to ignore remainder and stop iterating

[Stephen J. Turnbull]

Mathew Elman writes:

 > would it not be possible to have slicing fallback to islice if
 > __iter__ is implemented and __geitem__ is not?

The syntax is well-defined, but the semantics are not.

Consider "g[101]; g[100]" for g a generator object.  This either
requires all generators to keep a cache that can become arbitrarily
large, or violate the intuition of indexing by raising an error there,
or violate the intuition of indexing by returning the 102nd element,
then the 202nd element (what "islice(g,101,102); islice(g,100,101)"
would do), or some API to tell the generator to disable the cache if
you don't need it, or maybe there's some other semantics you could
give it that will be undesirable for some people in some other way.

Any of those makes 100% sense to me in the abstract, but I'm pretty
sure if it's made into syntax I'll be 99% dissatisfied with it. :-)
Explicit is better than implicit in this case.

You could argue that islice should be made a builtin, but I don't know
that it's used enough to justify that.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y773Y7MRIYIVQP6NLSAIG3MLK25EL6WA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Addition to fnmatch.py

[Stephen J. Turnbull]

Christopher Barker writes:

 > How do you divide a sequence into two sequences cleanly?

I don't know how to do it in a one liner, but

yes = []
no = []
for elem in seq:
(yes if test(elem) else no).append(elem)

seems pretty clean to me.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BBVBDT7KJTTOD5VQYIHBCQBU5QV32ZUB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add .except() and .only(), and .values_at(). instance methods to dict

[Stephen J. Turnbull]

David Mertz, Ph.D. writes:

 > These are all far too easy to do with comprehensions to merit new methods
 > or stdlib functions.

+1

 > E.g., we might provide additional set-like operators for dicts.

 > >>> m - {'a'}  # Would rightval be a set or dict though?!
 > >>> m & {'a', 'b'}  # Same question, but set feels better, I think

Why not "either"?  Of course if you allow dicts, the question in both
cases becomes whether the presence test is on keys or items.  I think
keys would be more frequently useful.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B5JREJ6C3QBF6WMKG3FW6YYUDPFGGH46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Efficiency of set difference

[Stephen J. Turnbull]

David Mertz, Ph.D. writes:

 > This is exactly the problem solved by set difference. E.g.
 > `{a, b, c} - {a, c}`.
 > 
 > This operation is linear on the size of the removed set.

Couldn't it be linear on min(len(left), len(right))?  Ie,

if len(left) < len(right):
for elem in left:
if elem in right:
   left.discard(elem)
else:
for elem in right:
left.discard(elem)

The upper loop is probably only about half as fast as the lower (two
hashes needed), so maybe the length test could be tuned.

The whole idea is probably not worth it, I can't think of practical
use cases at a scale where it would matter.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MOZZOH4ZMRKJ6X2BHY2RYMU7IHMJEFOF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Opt-in “iter def” and/or “gen def” syntax for generator functions

[Stephen J. Turnbull]

Serhiy Storchaka writes:

 > The advantage is that you cannot accidentally turn a function into a 
 > generator by adding "yield".

Can't mypy catch this?

 > Asynchronous functions are more reliable. "async" is mandatory, and if 
 > you do not await the result of an asynchronous function call you will 
 > get a loud warning.

"yield" is mandatory in generators, and it should be possible to
detect if a generator is called other than in an iteration context
(and possibly we would want to special-case functions like 'dir',
'help', and 'type' for this purpose).  As far as the signature goes, I
would suppose -> GeneratorType (or GeneratorType[T] if you know the
type of object to be yielded) would clue folks in, and mypy could warn
if it were missing.

Sure, you would like to have the guarantees of mandatory syntax, but
(unless I'm missing something) we can do this now.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RIUSDPCUVATHQD3HF6XYYSNZZ4FCR26R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python array multiplication for negative values should throw an error

[Stephen J. Turnbull]

fjwillemsen--- via Python-ideas writes:

 > Of course, such bugs do not occur solely due to the lack of a
 > raised errors on negative multiplication: in this case, a
 > combination of a faulty assumption on the programmers' part and
 > Python's lack of strict typing.

I'll take that bait: did mypy miss this?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BGNQTUXUGCGERMVZUAYZJIQQKOULRHGH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: TextIOBase: Make tell() and seek() pythonic

[Stephen J. Turnbull]

Chris Angelico writes:

 > If I'm reading this correctly, the result from f.tell() has enough
 > information to reconstruct a position within a hypothetical array
 > of code points contained within the file (that is to say - if you
 > read the entire file into a string, f.tell() returns something that
 > can be turned into an index into that string), but that position
 > might not actually correspond to a single byte location. Is that it?

That's what the OP wants.  That's not what f.tell does.  f.tell
returns information sufficient to recreate the state of the stream I/O
part of a codec when it reaches that point in the stream.  Its
*purpose* is to support producing the rest of the str produced by
f.read() after a shorter read, but f.tell doesn't care if the str ever
existed or ever will exist in the process that calls it.

 > I think UTF-7 is an awesome encoding. Really good at destroying
 > people's expectations of what they thought they could depend on.
 > 
 > (Terrible for actually using, though.)

Better than the alternatives for its intended use cases, which tells
you a lot about the hostile environment RFC-822 created for the rest
of the world. ;-)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U6RQ325GOS7232G2SGIOSN3IOZF2VH63/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] TextIOBase: Make tell() and seek() pythonic

[Stephen J. Turnbull]

mguin...@gmail.com writes:

 > There should be a safer abstraction to these two basic functions.

There is: TextIOBase.read, then treat it as an array of code units
(NOT CHARACTERS!!)

 > More details in the issue:

Not at all persuasive.  I'm with Chris: you need to present the
abstraction you want.

One thing you don't seem to understand: Python does *not* know about
characters natively.  str is an array of *code units*.  This is much
better than the pre-PEP-393 situation (where the unicode type was
UTF-16, nowadays except for PEP 383 non-decodable bytes there are no
surrogates to worry about), but Python doesn't care if you use NFD,
and there are characters that have no composed version (some are the
kind of thing you see in @jwz's display name on Twitter, but some of
them are characters that exist in national standards but not in
Unicode NFC form, I believe).

If code points are good enough for you, you need to specify that.

-- 
I, too, gruntle.  What about it?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ITJDGIE72WNWW7OW2ERL7Q4BBQO54RZN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Heterogeneous numeric data in statistics library

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > Users of the statistics module, how often do you use it with 
 > heterogeneous data (mixed numeric types)?

I don't use it often, but when I do the only coercion I ever need to
work is int -> float.  If I used anything else, I would convert first
anyway on the grounds of EIBTI.  That's also what I teach my students
(though I don't think I've ever seen a student use anything but int
and float).

Steve


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WDWZU4NNUKN37WNBSPWINY6N55OUIF42/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto assignment of attributes

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > What would this do?
 > 
 > def __init__(self, spam.x, eggs.y): pass
 > 
 > How about this?
 > 
 > def __init__(self, x, x.y): pass

IMO, both of those should be errors.  This syntax only makes much
sense for the first formal argument of a method definition, because
it's the only formal argument which has a fixed definition.  The form
"def foo(self, x, x.y)" has an interpretation, I guess, but

def foo(self, x, y):
x.y = y

is not a pattern I can recall ever seeing, and it could be relatively
easily relaxed if it were requested enough.  On the other hand, folks
do frequently request a way to DRY out long suites of "self.x = x"
assignments.

This could, of course, be done with a symbol such as '@' or even '.',
but '@' could also be used for other purposes (late binding, for
example), and "def foo(self, .x, .y):" looks like both grit on Tim's
screen and a typo.  On the other hand, I can't imagine what else might
be meant by "def foo(self, self.x):".

All that said, I'm not a fan of this feature as such.  But giving this
semantics to formal arguments of the form "self.x" is the most
intuitive (in the sense of "hard to give another interpretation") of
the proposals I've seen.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Z6DHYTG7F27IBER33UN7QOWZSFUQHUNL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto assignment of attributes

[Stephen J. Turnbull]

Devin Jeanpierre writes:

 > Is it unreasonable to instead suggest generalizing the assignment target
 > for parameters? For example, if parameter assignment happened left to
 > right, and allowed more than just variables, then one could do:
 > 
 > def __init__(self, self.x, self.y): pass

At a glance, this is the most attractive syntax I've seen for this.
If you're a keystroke minimizer, you can't do quite as well as

def __init__(self, @x, @y): pass

but

def __init__(s, s.x, s.y): pass

isn't bad at all.


 > 
 > Python 2 had non-variable parameters (but not attributes, just unpacking:
 > def foo((x, y)): pass), but it was removed in Python 3, because of issues
 > with introspection (among other, perhaps less significant, issues):
 > https://peps.python.org/pep-3113/
 > 
 > Perhaps now that positional-only parameters exist, and the introspection
 > APIs have evolved over time, there is a way to work this into the
 > introspection APIs sensibly.
 > 
 > (a "@foo" parameter does not have this problem, because it's just a
 > parameter named `foo` with some extra stuff.)
 > 
 > -- Devin
 > ___
 > Python-ideas mailing list -- python-ideas@python.org
 > To unsubscribe send an email to python-ideas-le...@python.org
 > https://mail.python.org/mailman3/lists/python-ideas.python.org/
 > Message archived at 
 > https://mail.python.org/archives/list/python-ideas@python.org/message/UY2TQTPRFAPN3OG3TBPCOYMAYZWFZ3E4/
 > Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KBWASQWSLY2TAOTU6KKJI433SGOFM2JX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

Stephen J. Turnbull writes:

 > It's usually helpful to assume that if you parse someone's statement
 > as nonsense, then probably you parsed it wrong.

Sorry, s/wrong/inconsistent with the statement's intended meaning/.

Obviously the statement was parsed correctly as English.  It's "wrong"
only in the sense that the conversation will usually move along more
quickly and productively if you try to reparse it meaningfully.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R54L3UGEYAIR4HSI4YUHX3HOOTMIRAMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

Steven D'Aprano writes:
 > On Sat, Apr 23, 2022 at 10:18:05PM +0900, Stephen J. Turnbull wrote:
 > > malmiteria  writes:
 > 
 > >  > If O1 and O2 are refactored into N1(GP) and N2(GP)
 > >  > the MRO as it was before refactoring was essentially N1, GP, N2, GP,
 > >  > as what was O1 before refactoring is equivalent to N1, GP after
 > >  > refactoring. 
 > >  > After refactoring, the MRO is now N1, N2, GP. Which do behave
 > >  > differently, in general.
 > > 
 > > Nobody denies that.
 > 
 > I denied the first part, and still do.

I meant "nobody denies that behavior under [N1, GP, N2] may differ
from that under [N1, N2, GP]".  I'm sure you don't deny that.

The point of malmiteria's "essentially", I assumed, was that the
sequences were representative search orders through the program text,
or something like that.  Not MROs, despite what was written there.

It's usually helpful to assume that if you parse someone's statement
as nonsense, then probably you parsed it wrong.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OARF47BZCLVXOYOPWAVROS3HW2KYEZ5U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

malmiteria  writes:

 > to give you exemples of problems :
 > 1) let's start with a django problem :
 > ```
 > class MyView(ModelView, PermissionMixin): pass
 > ```
 > 
 > Since it's mostly made out of django stuff, it's likely there
 > wouldn't be automated testing to check if the permissions are
 > indeed required when attempting to visit whatever resource MyView
 > serves. After all, automated testing should test your code, not the
 > library you're importing's code.

PermissionsMixin is documented to be *abstract* -- there are *no*
default permissions in it.  It either does nothing or errors when you
try to use the methods it declares.[1]

The permission-checking is (and must be) your code.  For one thing,
that's what "abstract" means, and in any case *permissions will be
defined by your application*, and permissions violations are an
application problem that the app devs should be testing for, if only
because ransomware is a billion-dollar industry.  Upstream is not
going to pay your ransom if some hacker walks in a door *you* left
wide-open and steals all your data.

I wish you'd stop making up non-problems and show us either working
code where you had to do something horrible (like use _as_parent)
because of the MRO, or non-working code where you have to restart from
scratch because of the MRO.  By "working" I mean it has some use other
than making the point that Python's use of the MRO sucks.

 > If O1 and O2 are refactored into N1(GP) and N2(GP)
 > the MRO as it was before refactoring was essentially N1, GP, N2, GP,
 > as what was O1 before refactoring is equivalent to N1, GP after
 > refactoring. 
 > After refactoring, the MRO is now N1, N2, GP. Which do behave
 > differently, in general.

Nobody denies that.

[...]
 > And no, class.method doesn't work in this case either.

I don't understand what you mean by that.  Here's a concrete
implementation of the case, where O1.a and O2.a don't even share any
code, and only N2 overrides a after refactoring a into a parent class
GP.

% python3.10
Python 3.10.4 (main, Mar 25 2022, 04:37:13) [Clang 13.0.0 (clang-1300.0.29.3)] 
on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class GP:
...  def a(s):
...   print('This is GP')
... 
>>> class N1(GP):
...  pass
... 
>>> class N2(GP):
...  def a(s):
...   print('This is N2')
... 
>>> class C(N1, N2):
...  def b(s):
...   print('super: ', super().a(s))
...   print('N1: ', N1.a(s))
...   print('N2: ', N2.a(s))
...   print('GP: ', GP.a(s), " # for completeness, the N1 case is the important 
one')
... 
>>> c = C()
>>> c.b()
super: This is N2
N1: This is GP
N2: This is N2
GP: This is GP # for completeness, the N1 case is the important one
>>> 

That's what I would want.  I don't have to go farther down the
hierarchy than the class I explicitly declared to get the result I
want.  What do you mean by "work"?  Or do you mean a different case?
If so, what is it?

Steve


Footnotes: 
[1]  It could be "abstract" in the sense that there is a standard way
to express permissions defined by the methods, but none are loaded.
Then it does nothing when its methods are invoked.  Or it could be
abstract in the sense of providing unimplemented methods, in which
case they will error.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/URP3XQ3566R5YJ7S5VIZQ5HSNKNNQ43P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

malmiteria  writes:
 > Stephen J. Turnbull writes

 > > Every feature means making an arbitrary choice that may or may
 > > not be what the programmers wanted.
 > 
 > I don't think that's what greg meant.

I don't either.  That's a separate comment that I made about the
nature of developing a product (namely, Python itself).  Greg wasn't
commenting on that, and he may or may not agree with me -- I'm pretty
sure he'll *understand* the point.  Whether he agrees doesn't matter
for the point I was making, which was about how *I* interpret "when
faced with uncertainty, refuse to guess."

 > they can take the time to learn C3 in depth. Which most people
 > don't, so they are left guessing.

So what?  It rarely matters, and if it does, it almost never does real
harm because people who depend on super in MI contexts generally do
know what they're doing and design for success.  If it does do harm,
somebody was being too tricky for their own good.

Sure, the too tricky somebody may be a library author (hello, Django)
in which case the person we're worried about is innocent collateral
damage.  I'm afraid that person is going to have to put out the effort
to learn what the MRO is and how it works, or consult someone who
already knows.  In any case, I don't see evidence that such collateral
damage is anything but a theoretical possibility.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L4M3N4TNGVS4F7HLGYQ5XT7ADKQGDKAE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

Greg Ewing writes:
 > On 16/04/22 10:46 pm, Steven D'Aprano wrote:

 > > There is no *guessing* in the C3 linearization algorithm.
 > 
 > "Guessing" in the context of that Zen line means making an arbitrary
 > choice that may or may not be what the programmer wants. It doesn't
 > mean choosing at random.

Every feature means making an arbitrary choice that may or may not be
what the programmers wanted.  That's why we sometimes rewrite stuff in
C: because most applications of "such stuff" we want time or space
performance more than we want the dynamic power of Python.  But we
have a requirement (at least a strong suggestion) that where possible
C modules should also provide Python implementations so that people
who want the power of Python can have it, at some cost in performance.
So in the context of the Zen, I suggest we "refuse to guess" *because*
that arbitrary choice also forecloses (or at least makes very
difficult) other choices.  Otherwise, that Zen would discourage adding
any features at all!

But to me, C3 is hardly arbitrary.  The technical sense in which a
deterministic algorithm doesn't guess is irrelevant to the Zen, as you
point out.  But I don't think that's what Steven meant.  Someone wrote
that C3 is based on 5 requirements, which could also be considered
assumptions about what the programmer wants.  I don't recall what they
are exactly, but they seemed really plausible to me as a guide to what
most programmers would want, or at least find acceptable, most of the
time.  If you're going to call something even in the event of multiple
definitions, C3 has benefits (I don't have time to enumerate them
right now).  I don't think that's an arbitrary guess, and I think
that's what Steven meant.

But still, what if that's not what the programmer wants?  Can they get
what they want, *as easily as if the feature didn't exist*?  How often
does it happen that they can't?  I haven't seen any compelling answers
to those questions, only cooked-up toy programs whose only known use
is to give the answer "yes" to the question "can I find a program
where super() and/or C3 can't do the thing I arbitrarily prespecify
that the program must do?", and one (1) hearsay story whose "harm" is
perfectly well redressed by "OK, don't fix what ain't broke, then".

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/63GGODAVSDZKSNK33SXYZZQFFLN7HY3F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Ethan Furman writes:
 > On 4/10/22 21:33, Stephen J. Turnbull wrote:
 > 
 > > I guess you could call the associative law of multiplication "dumb
 > > luck", but most mathematicians will consider that hate speech.
 > 
 > My apologies for not understanding your example.  The counter
 > example I had in my head, and should have written down,
 > was something like:
 > 
 >15mpg * 7l == how many miles?

Now it's my turn to not understand the point of this example.

Are you arguing Chris A's point that in some applications you want
those conversions done automagically[1], and in other applications you
want to get a YouSureYouMeanThatBoss? Exception[2]. :-)

Footnotes: 
[1]  American you just bought a car in Detroit MI and it still has the
EPA sticker "15mpg" on the window, and now you're in Windsor ON (Canada)
looking at the cost of the 7l you put in and wondering if you can get
back home to Ann Arbor MI on that.

[2]  Building a Mars lander for NASA.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/355ZIXMMVMCPPWB6M4D3TVR7R236Z4UF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

malmiteria  writes:

 > You'd need to return A.bar in C bar methods to get more symmetry,
 > but what if you wanted foo to visit the inheritance tree in this
 > order?

I don't want that, however.  Please give up this notion that you can
tell anybody else what they might want.  You need to show how that
alternative mro is useful, and for some purpose other than arguing
against the current design of super.

I wrote a lot more, but I need to cool down before sending it, if at
all.  But here's the general problem with all of your "what ifs":

Multiple inheritance is not a general specific, it does not solve all
problems.  "Some people, when faced with a problem, immediately think:
'I know, I'll use multiple inheritance!'  Now they have multiple
problems."

If multiple inheritance causes you problems, maybe it's absolutely the
wrong design pattern for your program.  So *you* need to show us
examples of real code for real problems[1] where there's a clear use
for multiple inheritance (or at least an API which documents that one
right way to use it is multiple inheritance) and still super() with
the C3 mro messes you up.


Footnotes: 
[1]  They don't have to be big problems or proprietary code; computing
Fibonacci sequences will do, if you can find a way to make MI relevant
to that task.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K2FYPXQEX4YAXNBXAKKRHJQSHV47RU7Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Warning: serious linguistic hacking follows.  I tried to be careful in
writing, please be careful in reading.  Corrections welcome.

Ethan Furman writes:
 > On 4/9/22 21:17, Stephen J. Turnbull wrote:
 > 
 > >  if 12*u.mm * 42*u.MFr == 502*u.foo:
 > >  print('Well done!')
 >
 > Part of the argument as well, I think, is that the top expression
 > would be parsed as:
 > 
 >  ((12 * u.m) * 42) * u.MFr
 > 
 > which, if actually equal to 502*u.foo, is dumb luck.

It's actually equal to 504*u.foo, I shouldn't do multiplications
larger than 12 * 12 in my head.  But yes, the correct answer is
(12 * 42) * u.foo, if I could only do arithmetic on integers!

I guess you could call the associative law of multiplication "dumb
luck", but most mathematicians will consider that hate speech.  If the
unit class is designed correctly (the units package is designed
correctly), both the associative and commutative laws of
multiplication hold.

The only gotcha as far as the value is concerned is that if you put in
divisions such as u.m/u.s, then you often will need parentheses.  But
that's true for ordinary arithmetic as well.  There would be a problem
if the LHS were 12*u.mm / 42*u.MFr.  That's actually nonsense in a
units-aware world.  Parentheses are required: 12*u.mm / (42*u.MFr).
The "we demand syntax" crowd wants the "add unit" operation to have
higher precedence than numerical multiplication and division.  Hmm,
unfortunately both '@' and '%' have the same precedence as '*' and
'/', but we could make the precedence more natural by using '**' at
the cost of the intuition that a unit is just a quantity (object with
both value and unit attributes) with value 1.

I'm not sure that intuition helps the "we demand syntax" crowd,
though.  Another way to put the issue is that libraries like units
don't provide units, they provide quantities.  You can create a
quantity foo that acts like a unit by giving it the value 1, but
10*foo is of the same type as foo.  It just has value attribute 10.
Again, that doesn't bother me, but I suspect that the "we demand
syntax" crowd will feel queasy about the idea that 'nm' *is* '1 nm'.
For them, 'nm' needs to be notated with a value to be a quantity.

I don't want to put words in anybody's mouth.  If Brian or Rickey
disclaims that interpretation, they're right.  If they think it has
expressive value but needs to be amended, they're right.  I'm just
trying to provide words to express what the need is here.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MQW2YFN7I2ATZV2AOPHVKLW2H6EU3FCI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > There are many things which are core to science and engineering but 
 > aren't part of the core Python language. What makes units of 
 > measurements more special than, say, numpy arrays or dataframes?

Arrays and dataframes are data structures, hidden behind the syntax.
They can be conveniently referenced and manipulated with functions
invoked via existing syntax.  This existing syntax is familiar to us
from many decades of programming language practice, and we have always
accepted it because of the constraint of 1-dimensional computer text.
Much of it is extensions of centuries of mathematical practice, from
operator symbols to function call notation, to these modern data
structures.  And that's one thing I love about Python, the ability to
use familiar syntax in analogous ways.

Applying this line of thought to units, the practice for centuries (at
least in English and Japanese) has been to append the unit after the
value.  But this is a syntax error in Python, and even the 'value[unit]'
dodge fails for literal numbers, while it probably isn't needed often
for variables (I would probably even consider it bad style).  We can
construct a Unit class, and use the multiplication operator to combine
a "bare" number with the unit, but this doesn't feel right to naive
users.  To me it feels like multiplying a number by a list.  What I
ideally want is two parts of a coherent whole, like real and imag.
(Note: This does not bother me in practice at all, I'm trying to
empathize with the folks whole say they can't stomach Python's
approach to units at all.)

Furthermore, there's another problem: units "should" have their own
namespace, almost like keywords.  But it's much more complicated than
keywords, because (1) there are many more potential collisions, both
between ordinary identifiers and units ('m' as an integer variable
vs. 'm' as a unit) and among units ('A' for ampere and 'A' for
Angstrom), and (2) the units namespace should be extensible.

Both of these issues (natural language practice and namespace) can be
most fully addressed with syntax that allows an optional identifier
expression to be placed directly after a literal or identifier, and
enforcing the semantics that this optional identifier expression must
be composed of registered units and only the allowed operations (* and
/, I guess).  Invalid operations would be SyntaxErrors, unregistered
identifiers would be RuntimeErrors.

I think that one way both issues can be addressed without syntax is to
take a package like units, add "repertoire attributes", so we can
write things like this

import units
u = units.si.modifiable_copy()# the SI repertoire is read-only
u.register_unit('frob', 'Fr', composed=u.foo/u.m,
doc='Frob is a devo unit invented by A. Panshin.')

if 12*u.mm * 42*u.MFr == 502*u.foo:
print('Well done!')

That would work fine for me.  But I can see why somebody who
frequently uses interactive Python as a scientific calculator would
prefer to write

if 12 m/s * 42 s == 502 m:
print('Well done!')

with the base SI repertoire (a dozen or so prefixes and 7 units) in
builtins.

As far as I can tell, that's the whole argument.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XVR423DHHSJ63T56KZ3LCHQUEHMRPJVF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Brian McCall writes:
 Steven d'Aprano writes:

 > > you have shown nothing to justify why unit support must be built
 > > into the language itself.
 > 
 > I did what I could, but I'm not going to try and justify any more.

That makes me sad, because everybody in the thread acknowledges that
improving the Python distribution's support for units is a good idea,
but nobody is as enthusiastic about getting it done as you.

Chris Barker's comments about multiple attractive library
implementations are well-taken, I think, but I also think that with
more focus on getting a satisfactory module into the stdlib, it would
be quite possible to pick one that doesn't rely on non-stdlib types
(so I guess astropy.units would be out).

That doesn't directly get you the literal syntax for units you focus
on, but if units are easier to use, more applications will use them,
and perhaps build up momentum for a syntax change.  And the syntax
change is useless without the library.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GWOWO4SB5SMKZNKYJXELWGP74LSYH3Y6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Chris Angelico writes:

 > > It's very difficult to get *any* syntax change in.  In particular,
 > > changing '_' from an identifier component to an operator for
 > > combining numeric literals would invalidate *tons* of code
 > > (including internationalization code that is the 0.454kg nearest
 > > my heart).  I can't imagine that being possible.
 > 
 > >>> 123_456
 > 123456
 > >>> 123_456_
 >   File "", line 1
 > 123_456_
 >^
 > SyntaxError: invalid decimal literal
 > 
 > Trailing underscore is currently invalid, so that would be fine.

I really meant "operator", not just syntax.  It was an extreme example.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WNR75C4FNFF5LUTMVFMFVLGOUSBOSYW7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

malmiteria  writes:

 > to dive into conceptual ideas a bit more:

I'm not sure why you assume that nobody knows this stuff, at least at
the extremely high and fuzzy level of your discussion.

Executive summary: I see a lot of theory in your posts, but it's very
difficult to tie it to *my* practice, and you don't go into any detail
about *your* practice that requires the features you propose.

 > inheritance is usually what we do when we mean *is a*

This really isn't very useful.  Much more useful would be

Class Cat should inherit from class Animal if all concrete
attributes of Animal are appropriate for Cat, and all abstract
attributes of Animal can be instantiated appropriately for Cat.
*** This is what we mean when we say a cat *is an* animal. ***
The more you reduce "all" to "most", the less appropriate
inheritance is.

 > In such a case with multiple *is a* there's multiple strategies to
 > blend in the multiple *is a*.

This is true.  Given the rarity of multiple inheritance, and
especially the rarity of cases where the C3 MRO "gets it wrong", you
need to do a lot more than just point out the multiplicity.  You have
to demonstrate a need to support other cases.

IMO, it's *not* important to support the cases where all of the parent
methods of the same name should be called by automatically calling all
of them.  In Python, it's very easy to implement this directly if
necessary.  On the other hand, if we default to calling all parent
methods of the same name, it's very difficult to work around that,
even impossible if you can't modify the parent classes.  And this is
subject to one the same complaint that you make about the MRO, as for
some methods you might want all parent methods called, and others you
want only one (or a larger proper subset).

 > Today's MRO implicitely overrides all method from the second parent
 > with method from the first parent.
 > Essentially, the first parent (in declaration order) is dominant on
 > all attributes.
 > This isn't very subtle, as we could want some attribute of each
 > parent to be dominant, while others to be recessive.

It may not be flexible, but it is quite useful.  In most cases it's
quite trivial to pick up the version from the second parent:

class C(A, B):
def foo(self, *args):
return B.foo(self, *args)
# we don't define bar() because we want C to inherit A's
# version

But in my experience it's rarely needed.  Mostly, there's a primary
parent class that provides most of the functionality, and there are
skeletal mixins which have one job each.  YMMV.

 > My "can't assume one parent is more specialised"

As I've pointed out before, Python makes no such assumption.  Python
*forces* you to specify the order of parent classes, and defines the
semantics so that each parent takes precedence over any that follow it
in the class declaration.  Python assumes that will Just Work for you.
If it doesn't, you have to curse Python and work around it.  Python
shrugs your curse right off, though.

The questions are

1.  Are there any "!#$%ing Python" cases that cannot be worked around?
2.  How frequent are the "!#$%ing Python" cases compared to the Just
Works cases?
3.  How painful are the workarounds?

AFAICS, you have consistently doubled down on the nonsense quoted
above, and have not even tried to answer any of questions 1-3 with
concrete examples of code somebody actually uses.  The exception is
your story about your colleague's issue with order of parents in one
class in one Django application.  The answers in that story as you
told it are

1.  Not here.
2.  This was not a "!#$%ing Python" case.
3.  No workarounds needed, just don't fix what ain't broke.

 > That's why my proposal for those cases is to allow inheritance to
 > be postponed after definition time, and set by the final child
 > class that inherits from those.
 > 
 > ```
 > class Sphynx(Hairless(Cat)): pass
 > ```

That doesn't "look like" a mixin, though.  That looks like a simple
inheritance chain, and if so the "(Cat)" is redundant and Hairless is
very badly named because it doesn't tell you it's a cat.  Sure, you
can tell me Hairless is a mixin applied to Cat, and Hairless does not
inherit from Cat, but I'm still going to WTF every time I see it.  Not
sure what this "postponed inheritance" is supposed to mean, but that's
not a good way to write it, I'm pretty sure.

Also, Sphynx is a pretty horrible example.  Given the diversity of the
animal kingdom, Animal is almost certainly a (very) abstract base
class, with almost all characteristics added by composition.  Even
within the Felidae, there is an awful lot of variation.  So I would
expect Hairless to not even exist, and for Sphynx to look like:

class Sphynx(Cat):
def __init__(self, *args):
super().__init__(*args)
self.hair = None

You could make Hairless a mixin class:

class Hairless:
def __init__(self, hair=None):

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

malmiteria  writes:

 > Stephen J. Turnbull writes:
 > > One really plausible example is given in Raymond's piece: a later
 > > version of the same library refactors a "monolithic" class as a child
 > > of one or more "private" classes that are not intended to be exposed
 > > in the public API, but your multiply-derived class *written before the
 > > refactoring* Just Works.  As far as I can see, super(), and maybe even
 > > the deterministic MRO, is needed to make that work.

 > I'm curious about this exemple, do you have a link to share so i
 > could have a look at the code / change in code? This could be a
 > good exercice.

The example is not code, it's a description of a case where you would
definitely not know the class hierarchy at execution time because it
changes after you release your code.  It's described in "Python's
super() considered super!"  But you don't need to read it, there are
no more details than what's quoted above.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7QBPZL4BOKYKODXW2NQYCHZC5BJUGSQR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Brian McCall writes:

 > Stephen J Turnbull and Paul Moore have asked why the "need" for
 > something other than a library (or perhaps a better library). There
 > are a number of examples that show simple unit calculations and
 > it's easy to argue based on those that nothing further is needed.

Nobody is arguing that, though.  We ask *because* we believe you
perceive need (and we don't put it in quotes, for exactly the reason
you give -- we do not define "need" as "if I don't get it I will
literally die").  You don't need to prove your need to Python core.

You need to show two things.  (1) The particular solution proposed
solves problems that already available approaches don't (including
"too inconvenient to consistently use in practice" which might apply
to the library approach vs the user-defined literal approach).  (2)
There are enough people with this need to outweigh the slight costs to
all Python users of maintaining the facility, documenting it, and
learning it, and possibly preventing a more valuable use of the same
syntax.

 > In regulated environments, risk analysis and mitigation is very
 > much affected by whether a feature has native support

That you say this after citing Paul of all people gave me a chuckle,
as he's probably the most prominent advocate of a comprehensive
standard library on these lists.  If anybody will be sympathic to your
needs on these grounds, he will.  (I don't expect you to know that,
but it might be useful to know you have a potential ally.)

It's important to distinguish between several levels of "native
support" here.

0.  Adding units to all numeric objects.

I don't think level 0 is a very good idea.  I think it was Mac
Lane who pointed out that the solution to a problem expressed in
mathematics starts by giving a formal statement of the problem,
manipulates the form according to syntactic rules, and finally
interprets the final form as a semantic solution to the problem.
Similarly, as long as the expected units of the result, and the
units output by a Python program are the same, the numbers don't
need to have units.  I suspect that from the point of view of
astropy, attaching units to ndarray rather than atomic numeric
types is optimal for this reason, not a workaround.

1.  Syntax, aka language support (the original suggestion of
user-defined literals fits here because either the way numbers are
parsed or the way '_' is parsed would have to change).

It's very difficult to get *any* syntax change in.  In particular,
changing '_' from an identifier component to an operator for
combining numeric literals would invalidate *tons* of code
(including internationalization code that is the 0.454kg nearest
my heart).  I can't imagine that being possible.  There was a fair
amount of pushback to using it as the non-capturing marker in the
match statement.  Changing its interpretation only in a numeric
literal is probably possible, since as far as I know it's
currently an error to follow a sequence of digits with _ without
interposing an operator or whitespace.

NOTE: The big problem I see with this is that I don't see any
practical way to use syntax for non-numeric types like ndarray.
The problem is that in, say, economics we use a lot of m x 2
arrays where the two columns have different units (ie, a quantity
unit and a $/quantity unit).  The sensible way to express this
isn't some kind of appended syntax as with numbers, but rather a
sequence of units corresponding to the sequence of columns.

If I'm right about that, the only time the literal would be used
syntax is when using Python interactively as a calculator.  Any
time you read from a string or file, you have to interpose a
parsing step anyway, so it makes sense to handle the construction
of quantities there.

2.  Built-in, that is pre-loaded in the interpreter.  These are
implemented as library functionality in the builtins module.

I would definitely be against level 2.  It would pollute the
builtin namespace to the great detriment of all non-numeric
applications.

3.  Modules in the standard library.  Always available, but must be
explicitly imported to be used.

I think adding a module that provides units for the stdlib numeric
types is a very interesting idea.  I doubt I'd ever use it, and
I'm not sure that the requirements can be well-enough specified at
the moment to go to +1 on it.  (As the Zen says, "Although never
is often better than *right* now.)

For me, level 4 (available on PyPI) is good enough.  I can think of
cases where it would be useful for class demos to be able to
interrogate a computation for its use of units, but that's about it.
So I depend on descriptions of proponents' use cases to sort out the
question of level 1 vs. level 3.

 >

[Python-ideas] Re: mro and super don't feel so pythonic

[Stephen J. Turnbull]

Chris Angelico writes:

 > I'm curious when you would ever be subclassing something from another
 > library without knowing its hierarchy.

When the class is a public API, no?  I'm not sure why this isn't
obvious, am I missing something?

One really plausible example is given in Raymond's piece: a later
version of the same library refactors a "monolithic" class as a child
of one or more "private" classes that are not intended to be exposed
in the public API, but your multiply-derived class *written before the
refactoring* Just Works.  As far as I can see, super(), and maybe even
the deterministic MRO, is needed to make that work.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N3PPY54NYP3S6JBL4WLBU2WHE3K4PFVW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

I'll start with some opinionated stuff, but possibly helpful attempts
to understand the requirements follows from about the middle of the post.

Ricky Teachey writes:

 > [Brian McCall] had it right in his first post that spurred this
 > discussion. I'll quote bits of it:
 > 
 > > ...I have spent a fair amount of my own time, and I have seen so
 > > many others' time wasted because command line or input fields do
 > > not include units, or the inputs field units are accidentally
 > > handled with different units, or units are not used at all.

This has zip-o to do with having quantity (number-with-unit) objects
as part of the language.  It's all about parsing natural language, or
providing the user with sufficient hints about the expected input.
Ie, it's UI/UX.

 > > I get the sentiment that Python, or programming languages in
 > > general, are not meant to deal with units.

I have no idea where this came from.  The plethora of units libraries
both on PyPI and in various folks' personal toolkits gives it the lie.

 > > Anyone who has taken science or engineering classes in college
 > > knows what happens when you turn in homework with missing units
 > > in your answers - zero credit. Anyone who has worked out
 > > complicated calculations by hand, or with the help of packages
 > > like "units"

Oops, we just let the cat out of the bag.  How hard it is to use the
units library?  What is hard about it?  Sure, it's an annoying amount
of setup, but do it once for your field and copy-paste it (or import
from your toolkit) and the only differences between using units and
having user-defined literals is typing * instead of _, and the
possible embarrassment of typing "cm_per_in = 1 * in / 1 * cm" and
getting units meters**2 (but you immediately know you did something
wrong).

 > > ...The lack of native language support for SI units is a problem for an
 > > entire segment of programmers.

This is the X Y problem.  We'll all concede that units support is nice
to have, and if you want to consider the lack of units support to be
existential, fine by us.  But you're going to have to do a tun of work
to show that it needs to be *native* support (ie, in the language),
that it requires new syntax.

 > > They need to take another step forward to say that EVERY number
 > > has a unit, including "unitless".

I disagree, but it's not open and shut -- I at least am open to
evidence of this claim.  Suppose we do need that.  Even so, it's not
clear to me that this shouldn't be handled the way we handle typing,
that is, by treating a quantity of 1 kg as being a different type from
1 m, and inferring types.  (Example below.)

 > The old engineering disciplines- mine (civil engineering), structural,
 > electrical, etc- are the next frontier in the "software eats the world"
 > revolution, and they desperately need a language with native units
 > support

Again, nobody has provided any foundation for the claim that they
*need* *native* support, except that C++ jumped off that bridge so
Python should too.

What follows are questions that need to be answered to implement the
feature well.  They shouldn't be taken as "YAGNI".

 >  Python SHOULD be that language we do this with. It is awesome in every
 > other way. But if it isn't DEAD SIMPLE to use units in python, it won't
 > happen.

What does "dead simple" mean?  Do you expect people to be using Python
as a calculator so that the majority of their typing involves literal
quantities (ie, numbers with explicit units)?  Or is the more common
use case writing

def foo(v: Volume, d: Density) -> Mass:
return v * d

and have an I/O functionality that can convert strings or csv files to
Volume and Density objects, and Mass objects have a nice __str___.
Then in the rare case where you need to specify a quantity (eg,
demoing at the interpreter prompt) you have to write

foo(2.34 * m * m * m, 0.06 * kg / (m * m * m))

Is that too "difficult"?  If it is, how about

m3 = m * m * m
foo(2.34 * m3, 0.06 * kg / m3)

By the way, this is possible right now, mypy will check it for you.
(Of course somebody must define the classes Volume, Density, and Mass,
and write the I/O and __str__ functions.  But only the I/O is at all
hard.)

 > a big big part is that using units is too hard.

I have no idea what you mean by "using units".  Abstractly, yes, but
what programs are you going to write?  Are they going to be littered
with literal quantities, as Brian's proposal for custom literals
suggests?  Or is it just important that when you operate on some
quantities, their units are checked for compatibility, and results can
be easily checked for appropriate units if the programmer wants to?
Do people often confuse moments, energy, and torque in their
computations so that we need to distinguish them in our unit
libraries, or does that kind of thing happen so rarely that cancelling
SI units and prefixes wherever possible would give good enough
results?  (Note that the "units as types" 

[Python-ideas] Re: Native support for units [was: custom literals]

[Stephen J. Turnbull]

Greg Ewing writes:

 > With my "no new syntax" suggestion there would be no question here --
 > the only way to write it would be
 > 
 >  height = 5 * ft + 4.5 * in

I'm very sympathetic to the "no new syntax" suggestion, but suppose I
wanted to know how many cm there are in an in:

cm_per_in = 1 * in / 1 * cm

Of course that's a silly mistake, but the (sole, IMO) advantage of the
original proposal is that you can't make that silly mistake.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UJPWU4KQCH3HCIVSFC7LUD7PSLC2PFWS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Custom literals, a la C++

[Stephen J. Turnbull]

Brian McCall writes:

 > In terms of the internal representation of units, the
 > representation that is use for machine calculations, there are only
 > 7 units that need to be supported.

According to you.  I would like dollars and shares to be supported as
well as perhaps "kg of apples" (vs "kg of oranges").

What is a unit?  It is an attribute of a quantity that says how it may
be combined with other quantities to produce more quantities.  In some
situations (determining whether I should have "driven the backroads so
I wouldn't get weighed") I'm happy to ignore the commodities and add
up the kg, but in others (making out the invoice) I don't want to
multiply price of one commodity by the quantity of another.

I'm sure there are other applications besides accounting and physics,
and each will have its own idiosyncratic constraints on combining
quantities, ie, its own units.

 > > Having a single interpreter-wide namespace for units will cause many 
 > > name collisions. I expect that units should be scoped like variables 
 > > are, with some variant of the LEGB (Local, Enclosing, Global, Builtin) 
 > > scoping rules in place.
 > 
 > Yes, yes, yes!

This is a job for ... Superlibrary!  IOW, I'm with David.

There may be a justification for supporting custom literals in the
language, but really, in this it's just syntactic sugar for
constructing an object of a derived class (ie, a number, unit couple)
and should be designed that way.

I think it's probably way overkill to support quantities (ie, numbers
with units) as a primitive type in the language.  For many
computations, a library will be fast enough.  For iterative
computation or large data sets, I would guess it's very rare to want
to check compatibility of units on every operation.  Rather, for
function calls you want to treat them like types of the actual
arguments, and for vectorized computations, normally the whole vector
will have the same units.  And even if you did treat quantity as a
primitive type, few existing programs are likely to ported to use the
feature (and if the guts are some ancient Fortran library, they can't
be, you'll have to strip the type information off the numbers before
operating on them anyway).

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UO6EEVY6KULGDBI5MKJAXNNUYCN6ESFO/
Code of Conduct: http://python.org/psf/codeofconduct/