from:"David Mertz"

Let's call the styles a tie.  Using the SOWPODS scrabble wordlist (no
currency symbols, so False answer):

>>> unicode_currency = {chr(c) for c in range(0x) if 
>>> unicodedata.category(chr(c)) == "Sc"}
>>> wordlist = open('/usr/local/share/sowpods').read()
>>> len(wordlist)
2707021
>>> %timeit any(unicodedata.category(ch) == "Sc" for ch in wordlist)
176 ms ± 1.75 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
>>> %timeit any(unicodedata.category(ch) == "Sc" for ch in set(wordlist))
17.8 ms ± 121 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
>>> bool(set(wordlist) & unicode_currency)
False
>>> %timeit bool(set(wordlist) & unicode_currency)
18 ms ± 216 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Of course, this is a small character set of 26 lowercase letters (and
newline as I did it).  A more diverse alphabet might tip the timing
slightly, but it's going to be a small matter either way.

On Fri, Jun 2, 2023 at 7:49 PM Chris Angelico  wrote:
>
> On Sat, 3 Jun 2023 at 09:42, David Mertz, Ph.D.  wrote:
> >
> > Yeah... oops. Obviously I typed the version in email. Should have done it 
> > in the shell. But you got the intention of set-ifying the characters in the 
> > large string.
>
> Yep. I thought of that as I was originally writing, but absent
> benchmarking data, I prefer the simplest way of writing something.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/BVPDSXXOCOWZ5G2THPB3ZVG6VPXDBE24/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q2N4ZJHEJN4XP4S43K5V3RPMHXDMOUOH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

Yeah... oops. Obviously I typed the version in email. Should have done it
in the shell. But you got the intention of set-ifying the characters in the
large string.

Yes on lies, damn lies, and benchmarks.

On Fri, Jun 2, 2023, 7:29 PM Chris Angelico  wrote:

> On Sat, 3 Jun 2023 at 08:28, David Mertz, Ph.D. 
> wrote:
> >
> > This is just bar talk at this point.  I think we've shown that this is
> > easy enough to do that programmers can roll their own.
> >
> > But as idle chat goes, note that in your code:
> >
> >set(unicodedata.category(ch) for ch in s)
> >
> > If `s` is a billion characters long, then we make a billion calls to
> > the `.category()` method.  Python calls are comparatively expensive,
> > even on well optimized data structures like strings.
> >
> > In my version:
> >
> > bool(set(s) & set(unicode_categories['Sc'])
> >
> > The billion characters are first reduced to a smallish set of hundreds
> > or thousands of distinct characters without needing method calls. Then
> > that is intersected with a smallish set of characters in the category.
> >
> > You could optimize your version, however, simply by using:
> >
> >set(unicodedata.category(set(ch)) for ch in s)
>
> Or perhaps:
>
> set(unicodedata.category(ch) for ch in set(s))
>
> But measure before considering this worthwhile.
>
> > Yours provides more information, since it lists all the categories.
> > But if you REALLY only care about one category, then you still have to
> > ask `'Sc' in set(unicodedata.category(set(ch)) for ch in s)`.  Which
> > is fine, that's not a hard question to ask.
>
> If you REALLY want to just check whether any category is there, you
> probably want something like:
>
> any(unicodedata.category(ch) == "Sc" for ch in s)
>
> which is completely different from what you were suggesting, and still
> doesn't require the string of all codepoints in the category.
>
> Point is, querying the string is almost always going to be more
> efficient than intersecting with the full gamut of that category.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/KMHZOQJQPILZD6Z3AKKRQXGHXVYFQPER/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FC64VVAITJTQLIHQYT2BUHSU64VXJXSC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

This is just bar talk at this point.  I think we've shown that this is
easy enough to do that programmers can roll their own.

But as idle chat goes, note that in your code:

   set(unicodedata.category(ch) for ch in s)

If `s` is a billion characters long, then we make a billion calls to
the `.category()` method.  Python calls are comparatively expensive,
even on well optimized data structures like strings.

In my version:

bool(set(s) & set(unicode_categories['Sc'])

The billion characters are first reduced to a smallish set of hundreds
or thousands of distinct characters without needing method calls. Then
that is intersected with a smallish set of characters in the category.

You could optimize your version, however, simply by using:

   set(unicodedata.category(set(ch)) for ch in s)

Yours provides more information, since it lists all the categories.
But if you REALLY only care about one category, then you still have to
ask `'Sc' in set(unicodedata.category(set(ch)) for ch in s)`.  Which
is fine, that's not a hard question to ask.

On Fri, Jun 2, 2023 at 5:36 PM Chris Angelico  wrote:
>
> On Sat, 3 Jun 2023 at 07:28, David Mertz, Ph.D.  wrote:
> >
> > Sure. That's fine. With a sufficiently long strings my code is faster, but 
> > for "typical" strings yours will be.
>
> Really? How? Your code has to build a set of every character in the
> string; mine builds a set of every category in the string. Set
> intersection won't be slower for a smaller set.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/5C7WSPFDJ4A6LRHL67N7UFPXGU4KI56O/
> Code of Conduct: http://python.org/psf/codeofconduct/

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5XXPVXLWZQXEQW7B35QIPXHJK7G4N6X7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

Sure. That's fine. With a sufficiently long strings my code is faster, but
for "typical" strings yours will be.

On Fri, Jun 2, 2023, 5:20 PM Chris Angelico  wrote:

> On Sat, 3 Jun 2023 at 07:08, David Mertz, Ph.D. 
> wrote:
> >
> > def does_string_have_currency_mark(s):
> > return bool(set(s) & set(unicode_categories['Sc'])
> >
> > def does_string_have_numeric_digit(s): ...
> >
> > ... and so on.  Those seem like questions one asks often enough. Not
> > every day, but more than never.
> >
>
> These questions are much better answered with the
> unicodedata.category() function. First figure out what categories your
> string has:
>
> cats = set(unicodedata.category(ch) for ch in s)
>
> And then check whether Sc is in that set, or whatever others you care
> about.
>
> This way, the set contains only the categories, not the characters;
> there's no reason to do set intersection with all of the characters.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3EK66S27AO2IFBWPOIJ6ABUEJ6C6W2YB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DJXKMQNXP4O23LHN43YAVL4XSWUSWMUT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

def does_string_have_currency_mark(s):
return bool(set(s) & set(unicode_categories['Sc'])

def does_string_have_numeric_digit(s): ...

... and so on.  Those seem like questions one asks often enough. Not
every day, but more than never.

On Fri, Jun 2, 2023 at 4:59 PM Chris Angelico  wrote:
>
> On Sat, 3 Jun 2023 at 06:54, David Mertz, Ph.D.  wrote:
> >
> > If we're talking PyPI, it would be nice to have:
> >
> > unicode_categories = {"Zs": [...], "Ll": [...], ...}
> >
> > For all the various categories.  It would just take one pass through
> > all the characters to generate it, but then every category would be
> > fast to access later.  On the other hand, it's a few lines of code
> > with a lazy import.  Probably not enough code to put on PyPI.
> >
>
> Question: What is the advantage of having this? What are the use-cases?
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/4ZFJWXPYS6TWU7XBA5G63RY5H4KGOSW2/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ALTCCL6LRXS75PDVSZBGS5RGOHXJLPFC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

If we're talking PyPI, it would be nice to have:

unicode_categories = {"Zs": [...], "Ll": [...], ...}

For all the various categories.  It would just take one pass through
all the characters to generate it, but then every category would be
fast to access later.  On the other hand, it's a few lines of code
with a lazy import.  Probably not enough code to put on PyPI.

On Fri, Jun 2, 2023 at 4:32 PM Marc-Andre Lemburg  wrote:
>
> On 01.06.2023 20:06, David Mertz, Ph.D. wrote:
> > I guess this is pretty general for the described need:
> >
> >>>> %time unicode_whitespace = [chr(c) for c in range(0x) if 
> >>>> unicodedata.category(chr(c)) == "Zs"]
>
> Use sys.maxunicode instead of 0x
>
> > CPU times: user 19.2 ms, sys: 0 ns, total: 19.2 ms
> > Wall time: 18.7 ms
> >>>> unicode_whitespace
> > [' ', '\xa0', '\u1680', '\u2000', '\u2001', '\u2002', '\u2003',
> > '\u2004', '\u2005', '\u2006', '\u2007', '\u2008', '\u2009', '\u200a',
> > '\u202f', '\u205f', '\u3000']
> >
> > It's milliseconds not nanoseconds, but presumably something you do
> > once at the start of an application.  Can anyone think of a more
> > efficient and/or more concise way of doing this?
>
> There isn't. You essentially have to scan the entire database for
> whitespacy chars.
>
> > This definitely feels better than making a static sequence of
> > characters since the Unicode Consortium may (and has) changed the
> > definition.
>
> Which was my point: including the above in a stdlib module wouldn't make
> sense, since it increases module load time (and possibly startup time),
> so it's better to generate a string and put this verbatim into the
> application.
>
> However, this would have to be part of the Unicode database update dance
> and whitespace is only possible category of chars which would be
> interesting. Digits or numbers are another, letter, linebreaks, symbols,
> etc. others:
>
> https://www.unicode.org/reports/tr44/#GC_Values_Table
>
> It's better to put this into the application in question or to have
> someone maintain such collections outside the stdlib in a package on PyPI.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Jun 02 2023)
>  >>> Python Projects, Coaching and Support ...https://www.egenix.com/
>  >>> Python Product Development ...https://consulting.egenix.com/
> 
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
> eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>  D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> https://www.egenix.com/company/contact/
>   https://www.malemburg.com/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PXQN4HVSM4ZQEHSQQCQDED3ABKFZX5ES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

2023-06-01 Thread David Mertz, Ph.D.

I guess this is pretty general for the described need:

>>> %time unicode_whitespace = [chr(c) for c in range(0x) if 
>>> unicodedata.category(chr(c)) == "Zs"]
CPU times: user 19.2 ms, sys: 0 ns, total: 19.2 ms
Wall time: 18.7 ms
>>> unicode_whitespace
[' ', '\xa0', '\u1680', '\u2000', '\u2001', '\u2002', '\u2003',
'\u2004', '\u2005', '\u2006', '\u2007', '\u2008', '\u2009', '\u200a',
'\u202f', '\u205f', '\u3000']

It's milliseconds not nanoseconds, but presumably something you do
once at the start of an application.  Can anyone think of a more
efficient and/or more concise way of doing this?

This definitely feels better than making a static sequence of
characters since the Unicode Consortium may (and has) changed the
definition.  In particular, MONGOLIAN VOWEL SEPARATOR (U+180E) was
removed from the whitespace category to which it previously belonged.
I'm not sure why U+FEFF isn't included, but that seems to match the
current standards, so all good.

On Thu, Jun 1, 2023 at 1:29 PM Marc-Andre Lemburg  wrote:
>
> On 01.06.2023 18:18, Paul Moore wrote:
> > On Thu, 1 Jun 2023 at 15:09, Antonio Carlos Jorge Patricio
> > mailto:antonio...@gmail.com>> wrote:
> >
> > I suggest including a simple str variable in unicodedata module to
> > mirror string.whitespace, so it would contain all characters defined
> > in CPython function
> > 
> > [_PyUnicode_IsWhitespace()](https://github.com/python/cpython/blob/main/Objects/unicodetype_db.h#L6314
> >  
> > )
> >  so that:
> >
> >   # existent
> > string.whitespace = ' \t\n\r\x0b\x0c'
> >
> > # proposed
> > unicodedata.whitespace = '
> > 
> > \t\n\x0b\x0c\r\x1c\x1d\x1e\x1f\x85\xa0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000'
> >
> >
> > What's the use case? I can't think of a single occasion when I would
> > have found this useful.
>
> Same here.
>
> For those few cases, where it might be useful, you can easily put the
> string into your application code.
>
> Putting this into the stdlib would just mean that we'd have to recheck
> whether new Unicode whitespace chars were added, every time the standard
> upgrades. With ASCII, this won't happen in the foreseeable future ;-)
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Jun 01 2023)
>  >>> Python Projects, Coaching and Support ...https://www.egenix.com/
>  >>> Python Product Development ...https://consulting.egenix.com/
> 
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
> eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>  D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> Registered at Amtsgericht Duesseldorf: HRB 46611
> https://www.egenix.com/company/contact/
>   https://www.malemburg.com/
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/REMDZ2SVFVOIDEJYX3VSB2WUZTQPTTLM/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3CH6FHG4BCXNBTF4LBZOYLRNHEKXCMYY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

2023-06-01 Thread David Mertz, Ph.D.

OK, fair enough. What about "has whitespace (including Unicode beyond ASCII)"?

On Thu, Jun 1, 2023 at 1:08 PM Chris Angelico  wrote:
>
> On Fri, 2 Jun 2023 at 02:27, David Mertz, Ph.D.  wrote:
> >
> > It feels to me like "split on whitespace" or "remove whitespace" are
> > quite common operations.  I've been frustrated a number of times by
> > settling for the ASCII whitespace class when I really wanted the
> > Unicode whitespace class.
> >
>
> They are indeed, quite common. It's a good thing Python makes those easy.
>
> >>> len("\u2000spam\u2001".strip())
> 4
> >>> "spam\u2002ham".split()
> ['spam', 'ham']
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/CJB356TCUPJ7DITRHQE6NPJ2ILWGYXZY/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OCEQ5W4QYO3AGNVNGNKXB2E3QFPZW3AO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

2023-06-01 Thread David Mertz, Ph.D.

It feels to me like "split on whitespace" or "remove whitespace" are
quite common operations.  I've been frustrated a number of times by
settling for the ASCII whitespace class when I really wanted the
Unicode whitespace class.

On Thu, Jun 1, 2023 at 12:20 PM Paul Moore  wrote:
>
> On Thu, 1 Jun 2023 at 15:09, Antonio Carlos Jorge Patricio 
>  wrote:
>>
>> I suggest including a simple str variable in unicodedata module to mirror 
>> string.whitespace, so it would contain all characters defined in CPython 
>> function 
>> [_PyUnicode_IsWhitespace()](https://github.com/python/cpython/blob/main/Objects/unicodetype_db.h#L6314)
>>  so that:
>>
>>  # existent
>> string.whitespace = ' \t\n\r\x0b\x0c'
>>
>> # proposed
>> unicodedata.whitespace = ' 
>> \t\n\x0b\x0c\r\x1c\x1d\x1e\x1f\x85\xa0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000'
>
>
> What's the use case? I can't think of a single occasion when I would have 
> found this useful.
> Paul
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/374AYVMYWOLB2Q3NH3NM6UMEBK6KIFSP/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OT7NREOQC4OHNXMFJCWCDOXBQ3Z34VXH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: @lazy decorator an alternative to functools.partial ?

2023-05-17 Thread David Mertz, Ph.D.

On Wed, May 17, 2023 at 2:22 PM Daniel Guffey 
wrote:

> I'm a bit dubious about the pypi suggestion as packages are being
> regularly poisoned with malware ( e.g. New KEKW malware infects
> open-source Python Wheel files via a PyPI distribution | SC Media
> (scmagazine.com)
> 
>  )
> and support issues keep happening with package management tools.
>

This is an absurd complaint.  For one, the PyPA dealt with that very
quickly.  But more relevantly, Toolz is a package with many years of
development by well-trusted people.  Yes, getting a brand new malware onto
PyPI is a danger, but that's a completely unrelated issue than using
well-established and signed packages from known people.

If you weirdly distrust PyPI, you can equally get the same thing via
GitHub... I guess unless you also distrust those repos.

It's not absurd to suggest a new decorator for the standard library. But "I
don't trust PyPI" isn't going to win you any support for the idea.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BT2RQ3VUTEHHWVZZPWJZT2CK2TTLUWKX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make ellipsis an indicator of missing implementation

2023-04-29 Thread David Mertz, Ph.D.

Moreover,  I have worked on a lot of code where ellipsis is used to mean
"this branch isn't implemented yet, but doing nothing isn't terrible."

I don't love that code.  I'd rather it use `pass` and a clarifying comment.
But this change would make all that existing code much more broken.

On Sat, Apr 29, 2023, 9:22 AM Al Sweigart  wrote:

> I agree. Giving punctuation-based syntax like ... a meaning that isn't
> obvious on reading the code makes for unreadable code. Whereas having
> `raise Exception ('TODO')` accomplishes the same thing but also allows
> English documentation. The ... here is open to interpretation, especially
> as a new feature to an old established language like Python.
>
> On Thu, Apr 27, 2023, 10:07 AM Antoine Rozo 
> wrote:
>
>> Raising an exception in that case would be a breaking change only for an
>> aesthetic preference.
>> Some codes use ellipsis as this in functions body and don't want it to
>> raise an error.
>> Also it's going to be confusing to understand why an exception is raised
>> in that place.
>>
>> Le jeu. 27 avr. 2023 à 15:50, haael  a écrit :
>>
>>>
>>> In examples in the web very often we see unfinished or partial code,
>>> where the ellipsis "..." is used in place of missing instructions.
>>>
>>> Idea: make ellipsis used in that context throw an exception, reminding
>>> the user that this code is work in progress.
>>>
>>> The exception could be called ToDoError, or WorkInProgressError, or
>>> simply EllipsisError, and could be derived from NotImplementedError.
>>>
>>>
>>> ```
>>>
>>> def my_fun():
>>> ... # todo
>>>
>>> my_fun()
>>>
>>>  > WorkInProgressError: Implementation of `my_fun` is not finished.
>>>
>>> ```
>>>
>>> This change could break some code, as for now ellipsis in the middle of
>>> code is silently ignored, but I don't think anybody seriously relies on
>>> that behavior.
>>>
>>> haael
>>>
>>> ___
>>> Python-ideas mailing list -- python-ideas@python.org
>>> To unsubscribe send an email to python-ideas-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-ideas@python.org/message/QAU7LB46VGVZYESJUI6RKM2UB3WQR5N4/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> Antoine Rozo
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/6RX2H4WBASSUIBYB62GFPCTYZ6S3WBMW/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3SURW3ATOFY3TYRQTBLUHLOKQTWNFWSX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CVFJFBHEXMR6WG32S4APOJFSN45MFZ73/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allowing `str.format` to format one or more parameters instead of all parameters

2023-04-27 Thread David Mertz, Ph.D.

It's actually much worse than making obviously buggy code pass silently. It
can also make uncommon bugs almost impossible to detect.

Some code looks like `mystring.format(**values)`. Whether values contains
the needed keys can vary at runtime now. It might be fine under most code
paths, but fail in other cases.

That's already somewhat hard to catch, but the proposed change of passing
silently disguises even the intermittent glimpse into the error.

On Thu, Apr 27, 2023, 9:05 AM Matthias Görgens 
wrote:

> What is the use case for this?
>
> Does it have any use case that's not already served by functools.partial?
>
> As far as I can tell, this proposal would turn buggy code that currently
> throws an obvious exception into code that silently does the wrong thing.
>
> This seems more appropriate for PHP or JavaScript (or C) than for Python.
>
>
> On Sat, 22 Apr 2023, 06:23 Samuel Muldoon, 
> wrote:
>
>> Consider the following string:
>>
>> x = r"\mathjax{{color}}{{text}}"
>>
>> string `x` contains two parameters named `color` and the other named `
>> text`.
>>
>> Currently, python requires that the string class method `str.format` contain
>> key-word arguments for *all *parameters, not just *one* parameter
>>
>> result = r"\mathjax{{color}}{{text}}".format(color = "blue" , text = 
>> "Spanish")
>>
>> result = "\mathjax{blue}{Spanish}"
>>
>> Can we change str.format so that it is possible to change only one string 
>> parameter, but leave the other parameters alone?
>>
>> # pfr is partially_formatted_result
>>
>> pfr= r"\mathjax{{color}}{{text}}".format(color = "blue")   # ERROR! 
>> missing parameter `text`
>>
>> result = r"\mathjax{{color}}{{text}}".format(text = "Spanish") # ERROR! 
>> missing parameter `color`
>>
>> The implementation requires fewer than ten lines of code in python, probably 
>> less than fifty lines in C, or a different implementation language.
>>
>> class str:
>> def format(self, **kwargs):
>> for key in kwargs:
>> x = "{"+key+"}"
>> ostr   = kwargs[key].join(self.split(x))
>> return ostr # output string
>>
>> As an example, everywhere a string contained `{text}` it will now say 
>> `*Spanish*` .
>>
>>
>> *Samuel Muldoon*
>>
>> *(720) 653 -2408*
>>
>> *muldoonsam...@gmail.com *
>>
>>
>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/2A6RTXKQ3LCMKDGDIPOX525O52KYQJS7/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3HHUQV35HQMU6AZWS2NIMFHFEJW3W3LO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AEFUTF7GTMHAGHF5G4BX7GIYBNQ6REBI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support of YAML in Python STDLIB

2023-03-13 Thread David Mertz, Ph.D.

In a similar vein, I'm guessing that if XML were new today, we'd have zero
APIs in the standard library to support it... rather than SIX!



On Mon, Mar 13, 2023 at 12:23 PM Christopher Barker 
wrote:

> I think the answer is that yaml is a pretty complex specification --
> powerful and flexible, but a bit of a mess :-(
>
> This is why toml, rather than yaml, was selected for package metadata
> specification (pyproject,toml) -- which then resulted in an stdlib toml
> package.
>
> At this point, I think you'd have a hard time adding something like that
> to the stdlib if it wasn't used in the core tools themselves. It's
> acknowledged that you can't get far without *some* external dependencies
> these days, so it's not critical to have all the batteries in Python, even
> if it's a pretty common use case.
>
> JSON is pretty ubiquitous these days, but I'm not sure even that would be
> added today given the current state of packaging.
>
> This is all just my opinion having monitored a LOT of these discussion
> over the years.
>
> -CHB
>
>
> On Mon, Mar 13, 2023 at 8:57 AM scott.fields--- via Python-ideas <
> python-ideas@python.org> wrote:
>
>> Though this has been discussed before, the last detailed discussion
>> appears to have been roughly 9 years ago.
>>
>> YAML is a technical standard widely used in various projects and at least
>> two major ones (Ansible and containers).
>>
>> Though PyYAML has become ubiquitous as the primary Python solution to
>> supporting YAML, I would think it time for Python to formally adopt this
>> technology in the core foundation.
>>
>> Is there still  a reason why this isn't being pursued as native
>> infrastructure in Python?
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/HU755XCIZZBCG6PYCMQQVEUMNUEOHBZQ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/2KSX52MCS75443FICN7QLRZBXNFJSQUD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MT5YBJV3OQ5YR6PAKFNCBQG4Y4Q7JQ6W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Ampersand operator for strings

2023-03-05 Thread David Mertz, Ph.D.

Is it really that much longer to write `f"{s1} {s2}"` when you want that?

Maybe a couple characters more total, but once you are in an f-string, you
can also do a zillion other things at the same time.

On Sun, Mar 5, 2023 at 10:42 PM Rob Cliffe via Python-ideas <
python-ideas@python.org> wrote:

> Tl;dr: Join strings together with exactly one space between non-blank
> text where they join.
>
> I propose a meaning for
>  s1 & s2
> where s1 and s2 are strings.
> Namely, that it should be equivalent to
>  s1.rstrip() + (' ' if (s1.strip() and s2.strip()) else '') +
> s2.lstrip()
> Informally, this will join the two strings together with exactly one space
> between the last non-blank text in s1 and the first non-blank text in s2.
> Example:  " bar " & "foo"==" bar
> foo"
>
> This operator is associative, so there is no ambiguity in expressions
> such as
>  s1 & s2 & s3
> There *is* a possible ambiguity in expressions such as
>  s1 & s2 + s3
> where the relative precedence of `&` and `+` matters when s2 consists
> solely of white space.  E.g.
>  " A " & "  " + " C" would evaluate
> to " A C"
> not to " A C"
> because `+` has a higher precedence than '&'.
>
> Utility:
>  In decades of experience with another language which had such an
> operator
>  (spelt differently) I have frequently found it useful for constructing
>  human-readable output (e.g. log output, debug/error messages,
> on-screen labels).
>
> Cognitive burden:
>  This would of course be one more thing to learn.
>  But I suggest that it is fairly intuitive that
>  s1 + s2
>  s1 & s2
>  both suggest that two strings are being combined in some way.
>
> Bars to overcome:
>  This change would require no fundamental change to Python;
>  just adding an `__and__ function to the str class.
>
> Backward compatibility:
>  Given that `str1 & str2` currently raises TypeError,
>  this change would be close to 100% backward-compatible.
>
> Alternative meanings:
>  As far as I know nobody has ever suggested an alternative meaning
> for `&` on strings.
>
> Bikeshedding:
> (1) I don't think it is important for the utility of this change
>  whether `&` strips off all whitespace, or just spaces.
>  I think it is better if it strips off all whitespace,
>  so that it can be understood as working similarly to strip().
> (2) The definition could be simplified to
>  s1.rstrip() + ' ' + s2.lstrip()
>  (leave an initial/final space when one string is whitespace only).
>  Again the operator would be associative.
>  Again I don't think this is important.
>
> Rob Cliffe
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/66XP7GY56XU7H3P52IJENLSWJFW53XIN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M7H3VZFEUJFZDO3BAAVUGXNKBH6WF4NA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Deprecate misleading escapes in strings

2023-02-16 Thread David Mertz, Ph.D.

Wow! That would break SO MUCH of the code I've written!  E.g.:

translate = {"el": "ἐπιστήμη", "en": "Knowledge", "zh": "知识"}



On Thu, Feb 16, 2023 at 9:54 AM Arusekk  wrote:

> Hi all!
>
> I was writing a tutorial on the distinction between bytes and strings
> and why it is important, when I saw the root cause.  People coming from
> C, Perl, Python 2 and similar languages tend to misinterpret "\x90" for
> b"\x90" often.  My idea is that Python could deprecate string literals
> containing any non-ASCII codepoints specified in any way different from
> unicode or unicode escapes (\u, \U, \N).
>
> (Actually I found that I started having the idea already back in 2021 on
> StackOverflow[1].  The question is an excellent example of what I mean.)
>
> I would not go so far to follow JSON (disallowing \x11 and \222 escapes
> completely), but while writing "\x00" or "\0" is useful and widely used,
> "\x99" (and especially "\777"!) is probably marginal and definitely less
> explicit than "\u0099" (in the Zen of explicit better than implicit).
> Byte strings do not treat b"\u00ff" as b"\xff".
>
> In the first part of implementing it, Python could raise a SyntaxWarning
> (or should it be DeprecationWarning? BytesWarning?), suggesting "\x99"
> to either become b"\u0099" or b"\x99", eventually promoting it to some
> equally helpful SyntaxError.  All of it could be hidden behind a feature
> like from __future__ import backslashes (one nice name I can think of).
>
> The new regular expression for octals would be \\[01]?[0-7]{1,2} and
> \\x[0-7][0-9A-Fa-f] for hexadecimals, hopefully not confusing anyone,
> and not much more complex than the old ones.
>
> In the meantime, probably between introducing a warning and changing it
> to become an error (the most reasonable timeline I can think of now),
> the default ascii() representation should eventually use the \u0099 form
> for all such codepoints, to keep the invariant of eval(ascii(x)) == x
> without syntax warnings.  repr() is also affected, but it is fortunately
> limited to the [\x80-\xa0\xad] range.  I mean [\u0080-\u00a0\u00ad] :-)
>
> Another timeline would be to change the repr first, initially hidden
> under an interpreter flag or environment variable, then officially
> deprecate it in the documentation, then introduce the error guarded by
> from __future__ import backslashes or another flag, then make the repr
> use \u by default, then add the warning and finally make it always raise
> an error.
> As a precedent, breaking repr() was not a dealbreaker when introducing
> randomized seeds (even repr({"a", "b"}) is now unpredictable).
>
> This would be of course a breaking change for a lot of unit tests, and
> stuff like pickle should probably support old syntax, delaying any such
> change until a new protocol comes (if it applies to the newest one ---
> quite sure it does not).  Such a breaking change must be used wisely.
> Other changes to octal escapes could be sneaked in, based on conclusions
> from the 2018 'Python octal escape character encoding "wats"' thread[2]
> (I like writing "\0" and "\4" though, just to make my opinion clear).
> If going the whole hog, the 2015 'Make non-meaningful backslashes
> illegal in string literals' thread[3] could be revived as well, maybe
> even with "\f\v" deprecated, "\e" = "\33" introduced and such.
>
> Please let me know what you think, what else could break, and is it
> useful anywhere else apart from my use case, and what similar problems
> you have.
>
> Cheers,
> Arusekk
>
> [1]: https://stackoverflow.com/q/64832281/3869724
> [2]:
>
> https://mail.python.org/archives/list/python-ideas@python.org/thread/ARBCIPEQB32XBS7T3JMKUDIZ7BZGFTL6/
> [3]:
>
> https://mail.python.org/archives/list/python-ideas@python.org/message/PJXKDJQT4XW6ZSMIIK7KAZ4OCDAO6DUT/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ITBFU4GPJJVXTHT57WNLASXKL4R4MPF5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MZ7TDS42GWR7Z2J6WAHLS3M4PKZQPO34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-20 Thread David Mertz, Ph.D.

Oh yeah. Good points! Do we need a PEP for str.upper() to grow an optional
'locale' argument? I feel like there are examples other than the Turkish
i's where this matters, but it's past my bedtime, so they aren't coming to
mind.

Maybe Koine Greek which had not adopted the miniscule/majuscule distinction
of post 10th century CE that modern Greek inherited. I feel like
`s.upper(locale='koine')` might sensibly account for this.

On Wed, Dec 21, 2022, 1:23 AM Chris Angelico  wrote:

> On Wed, 21 Dec 2022 at 17:20, David Mertz, Ph.D. 
> wrote:
> >
> > I'm on my tablet, so cannot test at the moment. But is `str.upper()`
> REALLY wrong about the Turkish dotless I (and dotted capital I) currently?!
> >
> > That feels like a BPO needed if true.
>
> It's wrong about the ASCII i and I, which upper and lower case to each
> other. There's no way for str.upper() to be told what language it's
> working with, so it goes with a default that's valid for every
> language except Turkish and its friends. This also means that
> lowercasing "İ" will give "i" which uppercases to "I", so it doesn't
> round-trip. There is no solution other than a language-aware case
> transformation.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/P7VDKYKVEHYQT4HKQJPLSCZIKVVWYF46/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F3VEWT2QEWJQ2F65EBLNYMW5KOFM7NI5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-20 Thread David Mertz, Ph.D.

I'm on my tablet, so cannot test at the moment. But is `str.upper()` REALLY
wrong about the Turkish dotless I (and dotted capital I) currently?!

That feels like a BPO needed if true.

On Wed, Dec 21, 2022, 1:04 AM Steven D'Aprano  wrote:

> On Wed, Dec 21, 2022 at 09:42:51AM +1100, Cameron Simpson wrote:
>
> > With str subtypes, the case that comes to my mind is mixing str
> > subtypes.
> [...]
> > So, yes, for many methods I might reasonably expect a new html(str). But
> > I can contrive situations where I'd want a plain str
>
> The key word there is *contrive*.
>
> Obviously there are methods that are expected to return plain old
> strings. If you have a html.extract_content() method which extracts the
> body of the html document as plain text, stripping out all markup, there
> is no point returning a html object and a str will do. But most methods
> will need to keep the markup, and so they will need to return a html
> object.
>
> HTML is probably not the greatest example for this issue, because I
> expect that a full-blown HTML string subclass would probably have to
> override nearly all methods, so in this *specific* case the status quo
> is probably fine in practice. The status quo mostly hurts *lightweight*
> subclasses:
>
> class TurkishString(str):
> def upper(self):
> return TurkishString(str.upper(self.replace('i', 'İ')))
> def lower(self):
> return TurkishString(str.lower(self.replace('I', 'ı')))
>
> That's fine so long as the *only* operations you do to a TurkishString
> is upper or lower. As soon as you do concatenation, substring
> replacement, stripping, joining, etc you get a regular string.
>
> So we've gone from a lightweight subclass that needs to override two
> methods, to a heavyweight subclass that needs to override 30+ methods.
>
> This is probably why we don't rely on subclassing that much. Easier to
> just write a top-level function and forget about subclassing.
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Q6JQVEUAQXGX6EMAFVGYGGF7ZENUSMRP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DXKVTYFTVKM6C2QO4DVFMP7R6XXJCQMF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-18 Thread David Mertz, Ph.D.

On Sun, Dec 18, 2022 at 8:29 PM Steven D'Aprano  wrote:

> > However, if you want to allow these types to possibly *do* something with
> > the strings inside (validate them, canonicalize them, do a security
> check,
> > etc), I think I like the other way:
> > class html(str): pass
> > class css(str): pass
>
> The problem with this is that the builtins are positively hostile to
> subclassing. The issue is demonstrated with this toy example:
>
> class mystr(str):
> def method(self):
> return 1234
>
> s = mystr("hello")
> print(s.method())  # This is fine.
> print(s.upper().method())  # This is not.
>

I'd agree to "limited", but not "hostile."  Look at the suggestions I
mentioned: validate, canoncialize, security check.  All of those are
perfectly fine in `.__new__()`.  E.g.:

In [1]: class html(str):
   ...: def __new__(cls, s):
   ...: if not "<" in s:
   ...: raise ValueError("That doesn't look like HTML")
   ...: return str.__new__(cls, s)


In [2]: html("Hello")


In [3]: html("Hello")
---
ValueErrorTraceback (most recent call last)
 in 
> 1 html("Hello")


 in __new__(cls, s)
  2 def __new__(cls, s):
  3 if not "<" in s:
> 4 raise ValueError("That doesn't look like HTML")
  5


ValueError: That doesn't look like HTML


I readily acknowledge that's not a very thorough validator :-).

But this much (say with a better validator) gets you static type checking,
syntax highlighting, and inherent documentation of intent.

I know that lots of things one can do with a str subclass wind up producing
a str instead.  But if the thing you do is just "make sure it is created as
the right kind of thing for static checking and editor assistance, I don't
care about any of that falling back.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AEQCVTJ2ABFQSQHWM62JOJQJI6UU675Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-18 Thread David Mertz, Ph.D.

Using a typing approach sounds like a fantastic idea.  Moreover, as Stephen
showed, it's easy to make Emacs utilize that, and as I showed, it's easy to
make vim follow that.  I've only written one tiny VS Code extension, but it
wouldn't be hard there either.  I'm not sure how one adds stuff to PyCharm
and other editors, but I have to believe it's possible.

So I see two obvious approaches, both of which 100% fulfill Emil's hope
without new syntax:

#1

from typing import NewType


html = NewType("html", str)
css = NewType("css", str)


a: html = html("Hello world")
b: css = css("h1 { color: #99; }")


def combine(h: html, c: css):
print(f"Combined page elements: {h} | {c}")


combine(a, b)  # <- good
combine(b, a)  # <- bad



However, if you want to allow these types to possibly *do* something with
the strings inside (validate them, canonicalize them, do a security check,
etc), I think I like the other way:

#2

class html(str): pass
class css(str): pass


a: html = html("Hello world")
b: css = css("h1 { color: #99; }")


def combine(h: html, c: css):
print(f"Combined page elements: {h} | {c}")


combine(a, b)
combine(b, a)


The type annotations in the assignment lines are optional, but if you're
doing something other than just creating an instance of the (pseudo-)type,
they might add something.  They might also be what your text editor decides
to use as its marker.

For either version, type analysis will find a problem.  If I hadn't matched
the types in the assignment, it would detect extra problems:

(py3.11) 1310-scratch % mypy tagged_types1.py
tagged_types1.py:13: error: Argument 1 to "combine" has incompatible type
"css"; expected "html"  [arg-type]
tagged_types1.py:13: error: Argument 2 to "combine" has incompatible type
"html"; expected "css"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)


Using typing.Annotated can also be used, but it solves a slightly different
problem.




On Sun, Dec 18, 2022 at 5:24 PM Paul Moore  wrote:

> On Sun, 18 Dec 2022 at 21:42, Christopher Barker 
> wrote:
>
>> On Sun, Dec 18, 2022 at 9:48 AM David Mertz, Ph.D. 
>> wrote:
>>
>>> In general, I find any proposal to change Python "because then my text
>>> editor would need to
>>> change to accommodate the language" to be unconvincing.
>>>
>>
>> Personally, I’m skeptical of any proposal to change Python to make it
>> easier for IDEs.
>>
>> But there *may* be other good reasons to do something like this. I’m not
>> a static typing guy, but it segg do me that it could be useful to subtype
>> strings:
>>
>> This function expects an SQL string.
>>
>> This function returns an SQL string.
>>
>> Maybe not worth the overhead, but worth more than giving IDEs hints SATO
>> what to do.
>>
>
> I believe typing has annotated types that could do this.
> Paul
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XCACWMITDR5YNBICCNONLUGZUYC3NFRV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-18 Thread David Mertz, Ph.D.

Well, obviously I have to come to the defense of vim as well :-).  I'm not
sure what year vim got the capability, but I suspect around as long as
emacs.

This isn't for exactly the same language use case, but finding a quick
example on the internet:

unlet b:current_syntaxsyntax include @srcBash syntax/bash.vim
syntax region srcBashHi start="..." end="..." keepend contains=@srcBash

unlet b:current_syntaxsyntax include @srcHTML syntax/html.vim
syntax region srcHTMLHi start="^...$" end="^...$" keepend contains=@srcHTML

This is easy to adapt to either the named function convention:
`html('Hello')` or to the
standardized-comment convention.

In general, I find any proposal to change Python "because then my text
editor would need to
change to accommodate the language" to be unconvincing.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6PMUCHFX6FG2IT2VHANPGSPX4GNBJAII/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-17 Thread David Mertz, Ph.D.

Just to be clear on my opinion. I think Emil's idea was 100% appropriate to
share on python-ideas, and he does a good job of showing where it works be
useful. Sure, a background search is nice, but not required.

That doesn't mean I *support* the idea. I take a very conservative attitude
towards language changes. I hope I've provided okay explanation of my
non-support, but it's NOT a criticism of Emil in any way.

That said, Jim Baker pitched his similar idea to my at last PyCon, and I
remember coming closer to feeling supportive. Maybe partially just because
I know and like Jim for a long time. But I think he was also suggesting
some extra semantics that seemed to move the needle in my mind.

On Sat, Dec 17, 2022, 2:27 PM Eric V. Smith via Python-ideas <
python-ideas@python.org> wrote:

> Jim Baker has been working on tagged strings, and Guido has a working
> implementation. See https://github.com/jimbaker/tagstr/issues/1
>
> I thought Jim had a draft PEP on this somewhere, but I can’t find it.
>
> --
> Eric
>
> On Dec 17, 2022, at 11:14 AM, e...@emilstenstrom.se wrote:
>
> Hi everyone!
>
> I'm the maintainer of a small django library called django-components.
> I've run into a problem that I have a language-level solution (tagged
> strings) to, that I think would benefit the wider python community.
>
> *Problem*
> A component in my library is a combination of python code, html, css and
> javascript. Currently I glue things together with a python file, where you
> put the paths to the html, css and javascript. When run, it brings all of
> the files together into a component. But for small components, having to
> juggle four different files around is cumbersome, so I've started to look
> for a way to put everything related to the component _in the same file_.
> This makes it much easier to work on, understand, and with fewer places to
> make path errors.
>
> Example:
> class Calendar(component.Component):
>template_string = ''
>css_string = '.calendar { background: pink }'
>js_string = 'document.getElementsByClassName("calendar)[0].onclick =
> function() { alert("click!") }'
>
> Seems simple enough, right? The problem is: There's no syntax highlighting
> in my code editor for the three other languages. This makes for a horrible
> developer experience, where you constantly have to hunt for characters
> inside of strings. You saw the missing quote in js_string right? :)
>
> If I instead use separate files, I get syntax highlighting and
> auto-completion for each file, because editors set language based on file
> type. But should I really have to choose?
>
> *Do we need a python language solution to this?*
> Could the code editors fix this? There's a long issue thread for vscode
> where this is discussed: https://github.com/Microsoft/vscode/issues/1751
> - The reasoning (reasonable imho) is that this is not something that can be
> done generally, but that it needs to be handled at the python vscode
> extension level. Makes sense.
>
> Could the vscode language extension fix this? Well, the language extension
> has no way to know what language it should highlight. If a string is HTML
> or CSS. PyCharm has decided to use a "special python comment" #
> language=html that makes the next string be highlighted in that language.
>
> So if just all editors could standardize on that comment, everything would
> work? I guess so, but is that really the most intuitive API to standardize
> around? If the next statement is not a string, what happens? If the comment
> is on the same line as another statement, does it affect that line, or the
> next? What if there's a newline in between the comment in the string, does
> that work?
>
> *Suggested solution*
> I suggest supporting _tagged strings_ in python. They would look like
> html''.
> * Python should not hold a list of which tagged strings it should support,
> it should be possible to use any tag.
> * To avoid clashes with current raw strings and unicode strings, a tag
> should be required to be at least 2 characters long (I'm open to other ways
> to avoid this).
>
> I like this syntax because:
> 1. It's clear what string the tag is affecting.
> 2. It makes sense when you read it, even though you've never seen the
> syntax before.
> 3. It clearly communicates which language to highlight to code editors,
> since you can use the language identifiers that already exist:
> https://code.visualstudio.com/docs/languages/identifiers#_known-language-identifiers
> - for single letter languages, which are not supported to avoid clash with
> raw strings and unicode strings, the language extension would have to
> support "r-lang" and "c-lang" instead.
> 4. It mimics the syntax of tagged string templates in javascript (
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#tagged_templates).
> So it has som precedent.
>
> (If desirable, I think mimicing javascript further and making tagged
> strings call a function with the tag's name, would

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-17 Thread David Mertz, Ph.D.

On Sat, Dec 17, 2022, 1:03 PM  wrote:

> > Moreover, there is no reason an editor could not have a capability to
> > "colorize any string passed to a function named foo()."  Perhaps with
> some
> > sort of configuration file that indicates which function names correspond
> > to which languages, but also with presets.
>
> This is an interesting idea. Some counter-arguments:
> * Anything that's hidden behind a config file won't be used except by very
> few. So, as you say, you need presets somehow.

I've been using vim long enough that I probably only edit .vimrc (or
correspondingly for neovim) every week or two.

I use VS Code much less, so when I do, I probably edit setting.json more
like once a day (when I'm using it)

But many editors in any cases, have friendly custom editors for some
elements of their configs.

Of course, if presets are fine, indeed users need not change them. Tagged
templates do EXACTLY ZERO to make this less of a concern.

If there was a chance this could happen, it would solve my problem nicely.
> For the reasons above, I don't think this will be acceptable to editors.
>

I could trivially implement this in a few lines within every modern editor
I am aware of. I bet you can do it for your editor with less than 2 hours
effort.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3KTEUPIUJYJAAF2RBE3RVGQDISKCNTWM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

2022-12-17 Thread David Mertz, Ph.D.

My impression whenever this idea is proposed is like Barry's.  The "win"
isn't big enough not simply to use named functions.

Balancing out the slight "win" is the much larger loss of adding additional
complexity to the Python language. New grammar, new parser, possibly new
semantics if tagged strings are more than exclusively decorative.  It's not
a *huge* complexity, but it's more than zero, and these keep adding up.

Python is SO MUCH less simple than it was when I learned it in 1998.  While
each individual change might have its independent value, it is now hard to
describe Python as a "simple language."

Moreover, there is no reason an editor could not have a capability to
"colorize any string passed to a function named foo()."  Perhaps with some
sort of configuration file that indicates which function names correspond
to which languages, but also with presets.  The details could be worked
out, and maybe even an informal lexicon could be developed in a shared way.

But all we save with more syntax is two character.  And the function style
is exactly what JavaScript tagged strings do anyway, just as a shorthand
for "call a function".  Compare:

header = html`Hello`
header = html("Hello")

If we imagine that your favorite editor does the same colorization inside
the wrapped string either way, how are these really different?

On Sat, Dec 17, 2022 at 12:01 PM Barry Scott  wrote:

>
>
> > On 17 Dec 2022, at 16:07, e...@emilstenstrom.se wrote:
> >
> > Hi everyone!
> >
> > I'm the maintainer of a small django library called django-components.
> I've run into a problem that I have a language-level solution (tagged
> strings) to, that I think would benefit the wider python community.
> >
> > *Problem*
> > A component in my library is a combination of python code, html, css and
> javascript. Currently I glue things together with a python file, where you
> put the paths to the html, css and javascript. When run, it brings all of
> the files together into a component. But for small components, having to
> juggle four different files around is cumbersome, so I've started to look
> for a way to put everything related to the component _in the same file_.
> This makes it much easier to work on, understand, and with fewer places to
> make path errors.
> >
> > Example:
> > class Calendar(component.Component):
> >template_string = ''
> >css_string = '.calendar { background: pink }'
> >js_string = 'document.getElementsByClassName("calendar)[0].onclick =
> function() { alert("click!") }'
> >
> > Seems simple enough, right? The problem is: There's no syntax
> highlighting in my code editor for the three other languages. This makes
> for a horrible developer experience, where you constantly have to hunt for
> characters inside of strings. You saw the missing quote in js_string right?
> :)
> >
> > If I instead use separate files, I get syntax highlighting and
> auto-completion for each file, because editors set language based on file
> type. But should I really have to choose?
> >
> > *Do we need a python language solution to this?*
> > Could the code editors fix this? There's a long issue thread for vscode
> where this is discussed: https://github.com/Microsoft/vscode/issues/1751
> - The reasoning (reasonable imho) is that this is not something that can be
> done generally, but that it needs to be handled at the python vscode
> extension level. Makes sense.
> >
> > Could the vscode language extension fix this? Well, the language
> extension has no way to know what language it should highlight. If a string
> is HTML or CSS. PyCharm has decided to use a "special python comment" #
> language=html that makes the next string be highlighted in that language.
> >
> > So if just all editors could standardize on that comment, everything
> would work? I guess so, but is that really the most intuitive API to
> standardize around? If the next statement is not a string, what happens? If
> the comment is on the same line as another statement, does it affect that
> line, or the next? What if there's a newline in between the comment in the
> string, does that work?
> >
> > *Suggested solution*
> > I suggest supporting _tagged strings_ in python. They would look like
> html''.
> > * Python should not hold a list of which tagged strings it should
> support, it should be possible to use any tag.
> > * To avoid clashes with current raw strings and unicode strings, a tag
> should be required to be at least 2 characters long (I'm open to other ways
> to avoid this).
> >
> > I like this syntax because:
> > 1. It's clear what string the tag is affecting.
> > 2. It makes sense when you read it, even though you've never seen the
> syntax before.
> > 3. It clearly communicates which language to highlight to code editors,
> since you can use the language identifiers that already exist:
> https://code.visualstudio.com/docs/languages/identifiers#_known-language-identifiers
> - for single letter languages, which are not supported to avoid

[Python-ideas] Re: Better (?) PRNG

2022-12-06 Thread David Mertz, Ph.D.

For background, Random123 was developed for a supercomputer that does
molecular dynamic simulations.  In particular, for the Anton supercomputer,
complete reproducibility of simulations was/is an important constraint.  In
concept, in that context, you might want to "jump to timestamp 1 billion,
and run forward from there" (based on a saved snapshot of the chemical
system at the timestamp).  In simulations, a certain element of
"reproducible randomness" is relevant to inject.

For that specific purpose, the splitting of generators that Matthias
mentions isn't especially helpful.  But for other purposes, it absolutely
is.  And yes, most certainly these two techniques could be complementary,
they need not be mutually exclusive.

On Tue, Dec 6, 2022 at 8:45 PM Matthias Görgens 
wrote:

> On Tue, 15 Nov 2022 at 00:14, David Mertz, Ph.D. 
> wrote:
>
>> In general, all PRNGs are deterministic, and by relying on a known seed,
>> the Nth element in a sequence of random numbers can always be
>> reconstructructed.  However, if a large number of random numbers are used,
>> certain replication scenarios make the purely sequential nature of
>> generators like Mersenne Twister or linear congruential generators
>> inconvenient.
>>
>> Counter based pseudo-random generators like Random123 use cryptographic
>> transformations upon counter variables, and were rigorously shown to pass
>> all standard tests of randomness of distribution.  However, these tests
>> were done using AES, Threefish, and Philox, and do not automatically apply
>> to SHA256 that James uses.  The advantage of these is that they allow
>> direct construction of the Nth element in a pseudo-random sequence without
>> large memory or CPU usage needed to construct the N-1 prior elements.
>>
>
>  There's another possibility that you haven't explored.
>
> You are only looking at random number generators that produce a linear
> sequence of numbers.
>
> If you add a 'split' function to your generator, that takes one generator
> and returns two generators that are independent of each other, you can
> build trees of random numbers, instead of just linear sequences.  Those
> trees also allow parallelisation.  (Implementations should take care to
> ensure that the resulting generators are not correlated. )
>
> You can combine the counter-based approach and the split based approach,
> of course.  If you have a cryptographic hash function, it's relatively easy
> to give a toy implementation.
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TNAJ2VBAHSFKFPASYJDVZF6F6BBTZBHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhancing variable scope control

2022-12-04 Thread David Mertz, Ph.D.

On Sun, Dec 4, 2022, 2:08 PM Chris Angelico  wrote:

> You're not the first to try to use globals() for this, but it means that
> the context manager works ONLY at top-level.

True. I know that. But if you're inside a function, you already have scope
containment, so the issue feels like "so what?"

But doh! I always forget about which way the locals() / globals()
mutability thing goes. A regular del without exec is better, of course.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V7ECRYFAH6WA532WAGX6NH3BWZHVU6UC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhancing variable scope control

2022-12-04 Thread David Mertz, Ph.D.

Like most commenters, I think the whole "create an anonymous function then
call it" scoping thing is too complex and has too many edge cases to be a
good idea.

That said, I decided to play around with what I can do to serve the general
purpose within existing Python:

>>> @contextmanager
... def local(pats=["scoped_*"]):
... try:
... yield
... finally:
... for _var in list(globals()):
... for pat in pats:
... if fnmatch(_var, pat):
... exec(f"del {_var}", globals())
...
>>> with local(['a', 'b', 'c']):
... a, b = 5, 6
... c = a + b
... d = c**2
... print(d)
...
121
>>> a
Traceback (most recent call last):
  Cell In [37], line 1
a
NameError: name 'a' is not defined

>>> d
121


Clearly, this still leaves it up to the programmer to make sure the "local"
names aren't already defined and important (i.e. with other values
already).  I guess you could make this more complex by copying globals(),
then restoring those that previously existed, even if they were defined
otherwise within the context.  But honestly, for the very limited purpose
desired, this implementation seems like plenty.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XCXHGD4QRNNDXSWTGHWAKJBIIE2KWZPB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhancing variable scope control

2022-12-01 Thread David Mertz, Ph.D.

I entirely agree with all of Steven's points.  I was going to reply with
similar comments, but his are precisely spot-on, and probably more detailed
than I would have provided.

In particular, the Avdi Grimm essay that Steven linked too is one of the
main references I would have provided as well.

Redefining methods at the instance level is somewhat dangerous.  But
redefining the classes themselves after definition—and especially doing so
for built-in classes—is where the greatest danger arises.  What happens is
that your `.TestDottedQuad()` becomes part of some package that is
generally useful, and some *other* package starts to build it in as a
dependency.  Who doesn't want a string to be able to test whether it's a
dotted quad, after all?

But then more packages build on that other package as well, and a whole
tower of dependencies lead back to your custom string method.  When you
realize that your method fails to handle some edge case correctly—or you
change your mind about what "correctly" means for this method—that whole
tower comes tumbling down because of the changed behavior.  Maybe
downstream pins to an old version of MousLib, but eventually that version
becomes unmaintained or incompatible with a later version of Python or some
other library.  Maybe the IANA or IETF updates a standard that factually
redefines what "dotted quad" means.  Maybe the method needs to grow an
optional parameter.

Or maybe you turn out to be the author of "left-pad" (
https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code
).

Writing a function `def test_dotted_quad(s str): ...` is perfectly
straightforward (and snake-cased to by more Pythonic).  But writing a class
that inherits from `str` is also perfectly plausible, and you can add
whatever methods you like.  Those instances of `MousString` are still
perfectly able to do all the string-ish things you'd like them to.  But we
nicely encapsulate "being able to do other things" within that class.

This danger isn't *absent* from the standard methods already built into
basic types.  The Unicode Consortium could make changes that makes
`mystring.title()` no longer behave the way it SHOULD under some edge cases
(maybe scholars decide the handling of illuminated capitals in Asomtavruli
is wrong in the Unicode 15.0 standard).  But if that occurs, there will be
PEPs, or at least substantial and careful discussion among the core
developers.

Yours, David...

On Thu, Dec 1, 2022 at 8:02 PM Steven D'Aprano  wrote:

> On Thu, Dec 01, 2022 at 03:19:38PM -0700, Anony Mous wrote:
>
> > I'd love to hear a justification -- any justification -- against what I'm
> > talking about, because to date, I've never run into one. Many have tried,
> > too. :)
>
> What is the `str` interface? Perhaps not the best example, because
> strings have so many methods, but it will do. Under Python's design, we
> know what methods strings have. We absolutely categorically know that if
> `type(obj) is str` then we can rely on a specific interface, namely 47
> methods and 33 dunders.
>
> What is the `str` interface with monkey-patching allowed? It's
> impossible to predict. The string interface depends on which methods
> have been monkey-patched in.
>
> It gets worse if we allow monkey-patching of individual instances as
> well as the entire class. Now you don't necessarily know that two
> strings support the same operations, even if their type is the same.
>
> If you have a string, you don't know whether or not it will support the
> TestDottedQuad method.
>
> These are points of friction that can make the language harder to use.
> How is a beginner supposed to know which methods are native to strings,
> and which are monkey-patched in? Where is `str.TestDottedQuad`
> implemented and documented?
>
> If you have a class and method you are not familiar with, say
> `Widget.frob()`, in a large application you don't know well, how do you
> find where `frob` was added to the class? It could have come from
> anywhere.
>
> What happens if you go to monkey-patch string with TestDottedQuad and
> *some completed unrelated library* has beat you to it and already done
> so? Monkey-patching is safe so long as you are the only one doing it. As
> soon as libraries get in the act, things go down hill very quickly.
>
> These are not insurmountable problems. Python supports powerful
> intraspection tools. Most classes that we write in pure Python, using
> the `class` keyword, support monkey-patching not just the class but
> individual instances as well, and it is considered a feature that most
> classes are extendable in that way. We mostly deal with that feature by
> *not using it*.
>
> The Ruby communittee learned the same lesson: the best way to use
> monkey-patching is to not use monkey-patching.
>
> https://avdi.codes/why-monkeypatching-is-destroying-ruby/
>
> So the ability to monkey-patch classes and instances is considered to be
> feature of marginal usefulness. Sure, sometimes

[Python-ideas] Re: Enhancing variable scope control

2022-12-01 Thread David Mertz, Ph.D.

This is very, very deliberately a design goal of Python.

Ruby is a largely similar language that deliberately made the decision you
advocate here. It's a good language, but also one in which monkey patching
by imported modules creates a huge amount of chaos.

Most large Ruby projects have adopted code standards to explicitly prohibit
the behavior you want, because the dangers greatly outweigh the benefits...
After concrete experience using said feature.

I think Python made the right decision here.

On Thu, Dec 1, 2022, 4:18 PM Anony Mous  wrote:

> >> the string class, which has some annoying tight couplings to "string"
> and  'string')
>
> > What does that mean?
>
> > It sounds like you are complaining that the syntax for creating strings
>  creates strings. Or possibly the other way around: that strings are
> created by string syntax. What did you expect?
>
> Here's my understanding:
>
> You can't extend the string class to add new functionality. You can only
> subclass it. Because of this, "character-strings" cannot be suffixed with
> new functionality.
>
> For instance, you can write x='FOO'.lower() and x ends up being "foo";
> That's a function of the actual str class.
>
> Buf if you subclass str as MyStr and create a MyStr object of
> "192.168.1.100", you cannot write...
>
> "192.168.1.100".TestDottedQuad()
>
> ...because "192.168.1.100" can only be an instance of str, not of MyStr.
> You have to run TestDottedQuad() either indirectly on a variable of type
> MyStr, or as a parameter to a function in a MyStr class object.
>
> The root problem is that you cannot extend a class. This is one of the
> limits that result. Which, yes, I find very annoying.
>
> So, some years ago, I wrote a simple precompiler that enables syntax that
> extends built-in classes -- without monkey patching Python. So you can
> create a virtual class extension, write x='192.168.1.100'.TestDottedQuad()
> or x="192.168.1.100".TestDottedQuad() in your input source to be
> precompiled, and it will work as it should in the resulting output source.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/4V36LWZTJ3GFNXPPBYYMY35SAC2Q2RNI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UDZV24FHTWGFEDNUT5ISETTL36KEEEIY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better (?) PRNG

2022-11-14 Thread David Mertz, Ph.D.

I should add, as does the Python documentation, that if you want genuinely
non-reproducible random values, the `secrets` module exists and produces
the best results possible on a given OS and computer architecture.  On
Unix-like systems, /dev/random is the best source of entropy you're going
to find, and trying to find your own is certainly going to be less good
than that source which `secrets` utilizes.

On Mon, Nov 14, 2022 at 11:13 AM David Mertz, Ph.D. 
wrote:

> In concept, what James suggests, is similar to the Random123 library,
> written at D.E.Shaw Research by my sadly late colleague John Salmon.  See
> https://github.com/DEShawResearch/random123 and the linked academic
> publications. E.g. https://dl.acm.org/doi/10.1145/2063384.2063405
>
> In general, all PRNGs are deterministic, and by relying on a known seed,
> the Nth element in a sequence of random numbers can always be
> reconstructructed.  However, if a large number of random numbers are used,
> certain replication scenarios make the purely sequential nature of
> generators like Mersenne Twister or linear congruential generators
> inconvenient.
>
> Counter based pseudo-random generators like Random123 use cryptographic
> transformations upon counter variables, and were rigorously shown to pass
> all standard tests of randomness of distribution.  However, these tests
> were done using AES, Threefish, and Philox, and do not automatically apply
> to SHA256 that James uses.  The advantage of these is that they allow
> direct construction of the Nth element in a pseudo-random sequence without
> large memory or CPU usage needed to construct the N-1 prior elements.
>
> That said, Random123 is *done right* by a number of very brilliant people
> who performed rigorous testing.  Something that "seems kinda similar" may
> or may not be of similar quality (more likely not).  FWIW, AES is built-in
> as a primitive instructruction on most modern general-purpose CPUs, so can
> be very fast to perform.
>
> On Mon, Nov 14, 2022 at 10:56 AM Barry  wrote:
>
>>
>>
>> > On 14 Nov 2022, at 14:31, James Johnson  wrote:
>> >
>> > 
>> > I wrote the attached python (3) code to improve on existing prng
>> functions. I used the time module for one method, which resulted in
>> disproportionate odd values, but agreeable means.
>> >
>> > I used the hashlib module for the second. It is evident that the code
>> is amateur, but the program might result in better PRN generation.
>> >
>> > The "app" lends itself to checking, using statistical tools (see
>> comments.)
>>
>> Have you used any cryptographic tools to prove the quality of your PRNG?
>> What results did you get?
>> How does your PRNG compare to what python already has?
>>
>> Without that this analysis this will be unlikely to be considered as a
>> candidate for python stdlib.
>>
>> Barry
>>
>> >
>> > I remain a fan,
>> >
>> > James Johnson
>> > 
>> > ___
>> > Python-ideas mailing list -- python-ideas@python.org
>> > To unsubscribe send an email to python-ideas-le...@python.org
>> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> > Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/VENAT2YTVYVHTBSEVHHMIURCU6MG2CEO/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/HWQV4AKQAUM7CY4NWS2IRIVLRAYMKR3V/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U6TW75PUJLKJN2HRU3WYWGGW7PNUQRST/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better (?) PRNG

2022-11-14 Thread David Mertz, Ph.D.

In concept, what James suggests, is similar to the Random123 library,
written at D.E.Shaw Research by my sadly late colleague John Salmon.  See
https://github.com/DEShawResearch/random123 and the linked academic
publications. E.g. https://dl.acm.org/doi/10.1145/2063384.2063405

In general, all PRNGs are deterministic, and by relying on a known seed,
the Nth element in a sequence of random numbers can always be
reconstructructed.  However, if a large number of random numbers are used,
certain replication scenarios make the purely sequential nature of
generators like Mersenne Twister or linear congruential generators
inconvenient.

Counter based pseudo-random generators like Random123 use cryptographic
transformations upon counter variables, and were rigorously shown to pass
all standard tests of randomness of distribution.  However, these tests
were done using AES, Threefish, and Philox, and do not automatically apply
to SHA256 that James uses.  The advantage of these is that they allow
direct construction of the Nth element in a pseudo-random sequence without
large memory or CPU usage needed to construct the N-1 prior elements.

That said, Random123 is *done right* by a number of very brilliant people
who performed rigorous testing.  Something that "seems kinda similar" may
or may not be of similar quality (more likely not).  FWIW, AES is built-in
as a primitive instructruction on most modern general-purpose CPUs, so can
be very fast to perform.

On Mon, Nov 14, 2022 at 10:56 AM Barry  wrote:

>
>
> > On 14 Nov 2022, at 14:31, James Johnson  wrote:
> >
> > 
> > I wrote the attached python (3) code to improve on existing prng
> functions. I used the time module for one method, which resulted in
> disproportionate odd values, but agreeable means.
> >
> > I used the hashlib module for the second. It is evident that the code is
> amateur, but the program might result in better PRN generation.
> >
> > The "app" lends itself to checking, using statistical tools (see
> comments.)
>
> Have you used any cryptographic tools to prove the quality of your PRNG?
> What results did you get?
> How does your PRNG compare to what python already has?
>
> Without that this analysis this will be unlikely to be considered as a
> candidate for python stdlib.
>
> Barry
>
> >
> > I remain a fan,
> >
> > James Johnson
> > 
> > ___
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/VENAT2YTVYVHTBSEVHHMIURCU6MG2CEO/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HWQV4AKQAUM7CY4NWS2IRIVLRAYMKR3V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/723TKSDJQUFV7KZFX6DIGUSKQJDBLSMS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Is it possible to provide an official type annotation checker？

2022-10-13 Thread David Mertz, Ph.D.

I happen to give a talk just yesterday that was about pretty much this (
https://github.com/DavidMertz/PythonRuntimeTyping; hopefully PyCon MEA
publishes video after a bit).

I illustrate checking the same program using mypy, pytype, and pyright.
There is a lot of overlap in what they flag as problems, but indeed there
are significant differences.  However, the differences are because of
different GOALS in the programs.

For example, mypy very explicitly only pays attention to code that is
explicitly annotated.  Pytype performs actual type inference as well, and
sometimes identifies problems even in completely non-annotated code; at the
same time, pytype deliberately ignores things that are "wrong" but that
will not affect runtime behavior.  Pyright is the most "aggressive" of
these tools (I haven't really played with pyre) but that is exactly
appropriate because the tool is mostly used as a plugin to VS Code (maybe
other editors), where the cost of a false positive complaint is small, and
the cost of a false negative is more important.

All three of these tools serve a reasonable, but different purpose.  Of
course they overlap to a large degree, but it would be bad to declare that
only one of these purposes was "legitimate."

On Fri, Oct 14, 2022 at 4:38 AM Christopher Barker 
wrote:

> Static type analysis is still experimental in Python.
>
> Evidence for this is that there ARE multiple type checkers, and they don’t
> all behave the same way.
>
> It really needs to settle down before there is a single “official” type
> checker.
>
> Also: Static type checking is optional in Python. There are those of us
> that are not convinced that static type checking is or should be a Python
> best practice. An official type checker would be an endorsement not just of
> that particular approach to type checking, but also the concept itself — I
> don’t think the community is ready for that.
>
> -CHB
>
>
>
> On Thu, Oct 13, 2022 at 9:23 AM  wrote:
>
>> I’m doing some research on Python type annotations recently.
>> I found that, some dynamic typing related bugs can be effectively avoided
>> in the early stages of code development,
>> by adding type annotations to Python code and applying some type checking
>> tools (such as MyPy, PyType, Pyright, and Pyre, etc.).
>> The introduction of type annotations effectively improves the type safety
>> of Python code.
>>
>> However, different tools discover, classify and report defects in
>> different ways, and there is little overlap in the defects they reported.
>> In the same code segment, some type checking tools will report defects
>> but some will not.
>> These may confuse users. Because it is not easy for the user to determine
>> which tools give credible results.
>>
>> Based on the above, I have the following suggestion:
>> Is it possible that Python provide a new type checker officially?
>> Or is it possible that Python choose one of the off-the-shelf type
>> checking tools as the official and recommended one?
>>
>> Python is my favourite programming language. I sincerely hope that I can
>> help Python become stronger and safer.
>> I hope this idea can be useful for you.
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/CG2IGZSV2Z4YMKLPK5MBWK4K4CDYTAFB/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UFEWUOXJNZDWFQLGXM6FFIO7EPAJEYHW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NZ5CP73IPBD4MVT7XBXX7NH5NLNUSUGW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add InvalidStateError to the standard exception hierarchy

2022-09-01 Thread David Mertz, Ph.D.

On Thu, Sep 1, 2022, 9:12 PM Chris Angelico

> > It depends on context whether it makes sense to define a custom
> exception, and I agree that I frequently should define a custom exception.
> In that case though, it would still be nice to have an appropriate generic
> exception for that to inherit from, just as I would inherit from
> `ValueError` for a special case of a value error.
>
>
> Can you describe a situation where you'd want to catch
> InvalidStateError from two unrelated libraries at the same time?
>
> TBH I'm actually a bit confused as to how you'd cope with
> InvalidStateError anyway. In what situations would you be able to recover
> from such an error (other than generic "log the error and continue", which
> works fine with "except Exception")?
>

I'm +0 on this. Maybe +0.5.

Making a custom error isn't hard, but inheriting from something more in the
ballpark is nicer.  Doing `except Exception ` feels very likely to be too
broad.

If my code hits ZeroDivisionError, I probably want to do something
different than I do for an ended subprocess or a closed file. Maybe they
all are just "log and continue," sure.

But for many resources, it's possible to require them, or at least
something functionally equivalent/usable. Maybe *that* subprocess ended,
but I might be able to launch another with suitable behavior. Maybe *that*
file handle is closed, but I can reopen the same file.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OX4L4SBH3GIHEL5XJ3GXGLCLMMMOCGR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make dataclass aware that it might be used with Enum

2022-07-10 Thread David Mertz, Ph.D.

I've seen this thread, and also wondered why anyone could EVER want a
dataclass that is an enum.  Nothing I've seen in the thread gives me any
hint about that, really.

On Sun, Jul 10, 2022 at 7:44 AM Barry Scott  wrote:

>
>
> > On 9 Jul 2022, at 22:53, Steve Jorgensen  wrote:
> >
> > I don't think that dataclasses have the limited set of intended uses
> that you are interpreting them as having. To me, the fact that they can be
> frozen makes them a good fit with Enum.
>
> Please quote the email that you are replying to.
>
> It is usually considered a code smell to have a class that is two or more
> things.
> This seems to be what you are trying to do.
>
> How can one class be a set of fields and also the enum for one of its own
> fields?
> I do not understand why this is resonable.
>
> Barry
>
>
>
> > ___
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/V6U7UMQRTLDZ2W6SWREL472L6ZH7MHB5/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HZFZE3YMCMT6CF5VZHNM6ZJVDAX6LZXK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HMYCB2RWSKK5SVKO2O2NF7OSHD4KZTR3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-24 Thread David Mertz, Ph.D.

On Fri, Jun 24, 2022 at 10:52 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> We (not sure how much help I'll be, but I'm in) need to deal with
> Chris A's point that a pure memoizing object doesn't help with the
> mutable defaults problem.  That is with
>
> def foo(cookiejar=defer []):
>
> foo() produces a late bound empty list that will be used again the
> next time foo() is invoked.
>

Yeah... this is a good point, and it is more frustrating to cover the
late-bound argument case.  I welcome edits to the proto-PEP (you can add
yourself as co-author, even if you wind up at -1 on it :-)).

I think I'm more-or-less U+1F922 on "deferred deferred" myself.

濫
-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GXVBVCYSTIIA7SVLJYWFDZRBEDRHSUCI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-24 Thread David Mertz, Ph.D.

On Fri, Jun 24, 2022 at 3:50 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> That's true in David's proposed semantics, where the runtime does that
> check.  I'm suggesting modified semantics where deferreds can be a
> proxy object, whose normal reaction to *any* operation (possibly
> excepting name binding) is
>
> 1.  check for a memoized value,
> if not found evaluate its stored code, and memoize the value
> 2.  perform the action on the memoized value
>

I think I like these semantics better than those my draft proposal.  I
haven't had a chance to enhance the proto-PEP more in the last few days
(other work).  But all of these comments are extremely helpful, and I'll
have a better version in a few days.  Hopefully I can address many of the
concerns raised.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2JUAXP53V35CEUTNDNV4UY2QRXR364XV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

Thanks Rob,

I recognize that I have so-far skirted the order-of-precedence concern. I
believe I have used parens in my example everywhere there might be a
question... But that's not a general description or rule.

I have a bunch of issues that I know I need to flesh out, many coming as
suggestions in this thread, which I appreciate. I just wanted to provide
something concrete to start the conversation.

FWIW, there is a bunch more at the link now than in my initial paste. But I
want to clarify more before I copy a new version into the email thread.

I haven't used Twisted in a while, but it is certainly an important
library, and I don't want to cause confusion. Any specific recommendation
on language to use?

On Wed, Jun 22, 2022, 8:45 PM Rob Cliffe via Python-ideas <
python-ideas@python.org> wrote:

> Thank you for your proposal David.  At last we have a counter-proposal to
> talk about.  A few points:
>
> (1) (As I pointed out in an earlier post) There is a flaw in using the
> syntax of an expression PRECEDED by a SOFT keyword:
> x =later-y
> With your proposal, x is assigned a deferred-evaluation-object which will
> be evaluated at some time later as "minus y", right?
> Erm, no.  This is already legal syntax for x being immediately assigned a
> value of "later minus y".
> If you put the soft keyword *after* the expression:
> x  =  -y  later
> it may or may not read as well (subjective) but AFAICS would work.
> Alternatively you could propose a hard keyword.  Or a different syntax
> altogether.
>
> (2) Delayed evaluation may be useful for many purposes.  But for the
> specific purpose of providing late-bound function argument defaults, having
> to write the extra line ("n = n" in your example) removes much of the
> appeal.  Two lines of boilerplate (using a sentinel) replaced by one
> obscure one plus one keyword is not much if any of a win, whereas PEP 671
> would remove the boilerplate altogether apart from one sigil.  Under your
> proposal, I for one would probably stick with the sentinel idiom which is
> explicit.  I think "n=n" is confusing to an inexperienced Python user.
> You may not think this is important.  My opinion is that late-bound
> defaults are important. (We may have to agree to differ.)  Apart from
> anything else: Python fully supports early-bound defaults, why discriminate
> against late-bound ones?
>
> (3) You talk about "deferred objects" and in one place you actually say 
> "Evaluate
> the Deferred".  A "deferred" is an important object but a different concept
> in Twisted, I think calling it something else would be better to avoid
> confusion.
>
> Best wishes
> Rob Cliffe
>
>
> On 21/06/2022 21:53, David Mertz, Ph.D. wrote:
>
> Here is a very rough draft of an idea I've floated often, but not with
> much specification.  Take this as "ideas" with little firm commitment to
> details from me. PRs, or issues, or whatever, can go to
> https://github.com/DavidMertz/peps/blob/master/pep-.rst as well as
> mentioning them in this thread.
>
> PEP: 
> Title: Generalized deferred computation
> Author: David Mertz 
> Discussions-To:
> https://mail.python.org/archives/list/python-ideas@python.org/thread/
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 21-Jun-2022
> Python-Version: 3.12
> Post-History:
>
> Abstract
> 
>
> This PEP proposes introducing the soft keyword ``later`` to express the
> concept
> of deferred computation.  When an expression is preceded by the keyword,
> the
> expression is not evaluated but rather creates a "thunk" or "deferred
> object."
> Reference to the deferred object later in program flow causes the
> expression to
> be executed at that point, and for both the value and type of the object to
> become the result of the evaluated expression.
>
>
> Motivation
> ==
>
> "Lazy" or "deferred" evaluation is useful paradigm for expressing
> relationships
> among potentially expensive operations prior their actual computation.
> Many
> functional programming languages, such as Haskell, build laziness into the
> heart of their language.  Within the Python ecosystem, the popular
> scientific
> library `dask-delayed `_ provides a framework for lazy
> evaluation
> that is very similar to that proposed in this PEP.
>
> .. _dask-delayed:
>https://docs.dask.org/en/stable/delayed.html
>
>
> Examples of Use
> ===
>
> While the use of deferred computation is principally useful when
> computations
> are likely to be expensive, the simple examples shown do not necess

[Python-ideas] Re: Generalized deferred computation in Python

On Wed, Jun 22, 2022 at 2:17 PM Eric V. Smith  wrote:

> Every time I’ve looked at this, I come back to: other than the clunky
> syntax, how is explicit evaluation different from a zero-argument lambda?

The difference is in composition of operations.  I can write a dozen
zero-argument lambdas easily enough.  But those are all isolated.

I've enhanced the PEP, so maybe look at the link for some of my updates;
but I need to add a bunch more, so don't want to repost each small draft
change.

But basically, think about `x = (later expensive1() + later expensive2()) /
later expensive3()`.  How can we make `x` itself be a zero argument
lambda?  And not just with those exact operations on the Deferreds, but
arbitrary combinations of Deferreds to create more complex Deferreds,
without performing the intermediate computations?

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y6LVDCFYU23XC6A7ZO2R7VUNTG344GC7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

On Wed, Jun 22, 2022 at 10:47 AM Martin Di Paola 
wrote:

> Hi David, I read the PEP and I think it would be useful to expand the
> Motivation and Examples sections.
> While indeed Dask uses lazy evaluation to build a complex computation
> without executing it, I don't think that it is the whole story.
> Dask takes this deferred complex computation and *plans* how to execute it
> and then it *executes* it in non-obvious/direct ways.
>

Dask is very clever about execution planning.  Ray is possibly even more
clever in that regard.

However, I think that that should be an explicit non-goal of the PEP.
DeferredObjects should create a DAG, yes.  But I think Python itself should
not think about being *clever* in evaluating that DAG, nor in itself think
about parallelism.  If my PEP were adopted, that would be something other
libraries like Dask or Django could build on top of with more elaborate
evaluation plans.

But just the DAG itself gets you more than just "wait until needed to do
the final computation." It allows for intermediate computation of nodes of
the DAG lower down than the final result.  For example, imagine
dependencies like this (where all the computation steps are expensive):

A -> B
 B -> Z
 B -> Y
 B -> X
A -> C
 C -> X
  X -> F
  X -> G
 C -> W
 C -> V
A -> D
 D -> V
 D -> U
 D -> T

Hopefully you either see my ASCII art in fixed font, or it's at least
intelligible.  If I want to force evaluation of A, I need to do
everything.  But if it turns out all I need within my program is C, then I
have to do computations C, X, F, G, W, V.  Which is maybe still expensive,
but at least I don't worry about B, Z, Y, U, T, or A.

Yes, I should add something like this to the PEP.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4CN3PRNH4M7T5TF36IRDZ6GZMHQS26TW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

Hi Martin,

Short answer: yes, I agree.
Slightly longer: I would be eternally grateful if you wish to contribute to
the PEP with any such expansion of the Motivation and Expansion.

On Wed, Jun 22, 2022 at 10:47 AM Martin Di Paola 
wrote:

> Hi David, I read the PEP and I think it would be useful to expand the
> Motivation and Examples sections.
>
> While indeed Dask uses lazy evaluation to build a complex computation
> without executing it, I don't think that it is the whole story.
>
> Dask takes this deferred complex computation and *plans* how to execute it
> and then it *executes* it in non-obvious/direct ways.
>
> For example, the computation of the min() of a dataframe can be done
> computing the min() of each partition of the dataframe and then
> computing the min() of them. Here is where the plan and the execution
> stages play.
>
> All of this is hidden from the developer. From his/her perspective the
> min() is called once over the whole dataframe.
>
> Dask's deferred computations are "useless" without the
> planning/execution plan.
>
> PySpark, like Dask, does exactly the same.
>
> But what about Django's ORM? Indeed Django allows you the build a SQL
> query without executing it. You can then perform more subqueries,
> joins and group by without executing them.
>
> Only when you need the real data the query is executed.
>
> This is another example of deferred execution similar to Dask/PySpark
> however when we consider the planning/execution stages the similarities
> ends there.
>
> Django's ORM writes a SQL query and send it to a SQL database.
>
> Another example of deferred execution would be my library to interact
> with web pages programmatically: selectq.
>
> Very much like an ORM, you can select elements from a web page, perform
> subselections and unions without really interacting with the web page.
>
> Only when you want to get the data from the page is when the deferred
> computations are executed and like an ORM, the plan done by selectq is
> to build a single xpath and then execute it using Selenium.
>
> So...
>
> Three cases: Dask/PySpark, Django's ORM and selectq. All of them
> implement deferred expressions but all of them "compute" them in very
> specific ways (aka, they plan and execute the computation differently).
>
> Would those libs (and probably others) do benefit from the PEP? How?
>
> Thanks,
> Martin.
>
> On Tue, Jun 21, 2022 at 04:53:44PM -0400, David Mertz, Ph.D. wrote:
> >Here is a very rough draft of an idea I've floated often, but not with
> much
> >specification.  Take this as "ideas" with little firm commitment to
> details
> >from me. PRs, or issues, or whatever, can go to
> >https://github.com/DavidMertz/peps/blob/master/pep-.rst as well as
> >mentioning them in this thread.
> >
> >PEP: 
> >Title: Generalized deferred computation
> >Author: David Mertz 
> >Discussions-To:
> >https://mail.python.org/archives/list/python-ideas@python.org/thread/
> >Status: Draft
> >Type: Standards Track
> >Content-Type: text/x-rst
> >Created: 21-Jun-2022
> >Python-Version: 3.12
> >Post-History:
> >
> >Abstract
> >
> >
> >This PEP proposes introducing the soft keyword ``later`` to express the
> >concept
> >of deferred computation.  When an expression is preceded by the keyword,
> the
> >expression is not evaluated but rather creates a "thunk" or "deferred
> >object."
> >Reference to the deferred object later in program flow causes the
> >expression to
> >be executed at that point, and for both the value and type of the object
> to
> >become the result of the evaluated expression.
> >
> >
> >Motivation
> >==
> >
> >"Lazy" or "deferred" evaluation is useful paradigm for expressing
> >relationships
> >among potentially expensive operations prior their actual computation.
> Many
> >functional programming languages, such as Haskell, build laziness into the
> >heart of their language.  Within the Python ecosystem, the popular
> >scientific
> >library `dask-delayed `_ provides a framework for lazy
> >evaluation
> >that is very similar to that proposed in this PEP.
> >
> >.. _dask-delayed:
> >   https://docs.dask.org/en/stable/delayed.html
> >
> >
> >Examples of Use
> >===
> >
> >While the use of deferred computation is principally useful when
> >computations
> >are likely to be expensive, the simple examples shown do not necessarily
> use
> >such expecially spe

[Python-ideas] Re: Generalized deferred computation in Python

Thanks Carl and Chris. After reading your comments, and thinking some more
about it, I agree you are both correct that it only makes sense to have a
DeferredObject act like a closure in the scope of its creation. That
really should suffice for anything I could sensibly want; it's enough for
Dask, and it's enough for functional programming languages with laziness.

Moreover, the limited laziness that Python currently has also follows that
rule. I also haven't gotten to writing that needed section of the PEP, but
I need to address generators and boolean shortcutting as limited kinds of
laziness. Obviously, my PEP is much broader than those, but generators
also use lexical scope not dynamic scope.

Possibly in my defense, I think Carl's PEP 690 can do the same thing. :-)
>
>
> I’m not sure what you mean. I don’t think there’s any way PEP 690 can
> introduce dynamic scoping like this. Can you give an example?
>

Let me just withdraw that. I think I might be able to construct something
where a module's top-level code does something perverse with the call
stack where it winds up getting evaluated that can act like dynamic
scoping. I might be wrong about that; but even if it's technically true,
it would need very abnormal abuse of the language (as bad as the lovely
zero-argument super() does :-)).

I don't think this actually has much effect on encompassing late-binding of
default arguments, other than needing to say the arguments contribute to
the scope on a left-to-right basis. But actually, despite writing the
section because of recent discussion, PEP-671 has little to do with why I
want generalized deferreds. I've suggested it on a number of occasions
long before PEP-671 existed (but admittedly always as a passing thought,
not as a detailed proposal... which my draft still is not either).

On one small point Chris mentions, I realized before he commented that
there was no need to actually rebind the deferred argument to force
evaluation of its default. Simply mentioning it would force the
evaluation. I found something similar in PEP-690 that reminded me that
doing this is plenty:

def func(items=[], n=later len(items)):
n # Evaluate the Deferred
items.append("Hello")
print(n)

Of course, that's an extra first line of the function for the motivating
case of emulating the sentinel pattern. But it also does let us *decide*
when each deferred gets "fixed in place". I.e. maybe a, b, and c are all
`later` arguments. We could make the first line `a, b` but only reference
`c` at some later point where it was appropriate to the logic we wanted.

--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/VYWVQZYB56C3OUYIUSEMZE7NEOX4UKXM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

I haven't gotten to writing that into the PEP yet, but I think the rule has
to be to take the scope of evaluation not the scope of definition. I know
it needs to be there,  and I'm just thinking about helpful examples.

Which is to say the semantics are more like `eval()` than like a lambda
closure.

... and I know this is going to raise the neck hairs of many folks,
because effectively I'm proposing a kind of dynamic scoping. Possibly in my
defense,  I think Carl's PEP 690 can do the same thing. :-)

On Tue, Jun 21, 2022, 6:27 PM Chris Angelico

> What are the scoping rules for deferred objects? Do they access names
> where they are evaluated, or where they are defined?
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ONBSI7GFDSGYEPMGHKA3OTY4S2VUPUJ4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

On Tue, Jun 21, 2022 at 5:53 PM Brendan Barnwell 
wrote:

> > In the example, we assume that the built-in function `type()` is special
> > in not counting as a reference to the binding for purpose of realizing a
> > computation. Alternately, some new special function like `isdeferred()`
> might be used to
> > check for ``Deferred`` objects.
>
> I'll have to ponder my thoughts about the proposal as a whole, but this
> particular aspect seems dubious to me.  As I understand it this would
> require some fairly deep changes to how evaluation works in Python.
> Right now in an expression like `type(blah)`, there isn't any way for
> the evaluation of `blah` to depend on the fact that it happens to occur
> as an argument to `type`.


I absolutely agree that this is a sore point in my first draft.  I could
shift the magic from `type()` to `isdeferred()`, but that doesn't really
change anything for your examples.  I suppose, that is, unless
`isdeferred()` becomes something other than a real function, but more like
some sort of macro.  That doesn't make me happy either.

However, I *would* like to be able to answer the question "Is this object a
DeferredObject?" somehow.  For example, I'd like some way to write code
similar to:

if isdeferred(expensive_result):
log.debug("The computationally expensive result is not worth
calculating here")
else:
log.debug(f"We already hit a path that needed the result, and it is
{expensive_result}")

Any thoughts on what might be the least ugly way to get that?


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IWCK3DRQDUASJ3ZB4ONMTT3FWHCWGC5L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Generalized deferred computation in Python

Here is a very rough draft of an idea I've floated often, but not with much
specification.  Take this as "ideas" with little firm commitment to details
from me. PRs, or issues, or whatever, can go to
https://github.com/DavidMertz/peps/blob/master/pep-.rst as well as
mentioning them in this thread.

PEP: 
Title: Generalized deferred computation
Author: David Mertz 
Discussions-To:
https://mail.python.org/archives/list/python-ideas@python.org/thread/
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 21-Jun-2022
Python-Version: 3.12
Post-History:

Abstract


This PEP proposes introducing the soft keyword ``later`` to express the
concept
of deferred computation.  When an expression is preceded by the keyword, the
expression is not evaluated but rather creates a "thunk" or "deferred
object."
Reference to the deferred object later in program flow causes the
expression to
be executed at that point, and for both the value and type of the object to
become the result of the evaluated expression.


Motivation
==

"Lazy" or "deferred" evaluation is useful paradigm for expressing
relationships
among potentially expensive operations prior their actual computation.  Many
functional programming languages, such as Haskell, build laziness into the
heart of their language.  Within the Python ecosystem, the popular
scientific
library `dask-delayed `_ provides a framework for lazy
evaluation
that is very similar to that proposed in this PEP.

.. _dask-delayed:
   https://docs.dask.org/en/stable/delayed.html


Examples of Use
===

While the use of deferred computation is principally useful when
computations
are likely to be expensive, the simple examples shown do not necessarily use
such expecially spendy computations.  Most of these are directly inspired by
examples used in the documentation of dask-delayed.

In dask-delayed, ``Delayed`` objects are create by functions, and operations
create a *directed acyclic graph* rather than perform actual computations.
For
example::

>>> import dask
>>> @dask.delayed
... def later(x):
... return x
...
>>> output = []
>>> data = [23, 45, 62]
>>> for x in data:
... x = later(x)
... a = x * 3
... b = 2**x
... c = a + b
... output.append(c)
...
>>> total = sum(output)
>>> total
Delayed('add-8f4018dbf2d3c1d8e6349c3e0509d1a0')
>>> total.compute()
4611721202807865734
>>> total.visualize()

.. figure:: pep--dag.png
   :align: center
   :width: 50%
   :class: invert-in-dark-mode

   Figure 1.  Dask DAG created from simple operations.

Under this PEP, the soft keyword ``later`` would work in a similar manner to
this dask.delayed code.  But rather than requiring calling ``.compute()``
on a
``Delayed`` object to arrive at the result of a computation, every
reference to
a binding would perform the "compute" *unless* it was itself a deferred
expression.  So the equivalent code under this PEP would be::

>>> output = []
>>> data = [23, 45, 62]
>>> for later x in data:
... a = later (x * 3)
... b = later (2**x)
... c = later (a + b)
... output.append(later c)
...
>>> total = later sum(output)
>>> type(total)  # type() does not un-thunk

>>> if value_needed:
... print(total)  # Actual computation occurs here
4611721202807865734

In the example, we assume that the built-in function `type()` is special in
not
counting as a reference to the binding for purpose of realizing a
computation.
Alternately, some new special function like `isdeferred()` might be used to
check for ``Deferred`` objects.

In general, however, every regular reference to a bound object will force a
computation and re-binding on a ``Deferred``.  This includes access to
simple
names, but also similarly to instance attributes, index positions in lists
or
tuples, or any other means by which an object may be referenced.


Rejected Spellings
==

A number of alternate spellings for creating a ``Deferred`` object are
possible.  This PEP-author has little preference among them.  The words
``defer`` or ``delay``, or their past participles ``deferred`` and
``delayed``
are commonly used in discussions of lazy evaluation.  All of these would
work
equally well as the suggested soft keyword ``later``.  The keyword ``lazy``
is
not completely implausible, but does not seem to read as well.

No punctuation is immediately obvious for this purpose, although surrounding
expressions with backticks is somewhat suggestive of quoting in Lisp, and
perhaps slightly reminiscent of the ancient use of backtick for shell
commands
in Python 1.x.  E.g.::

might_use = `math.gcd(a, math.factorial(b))`


Relationship to PEP

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

I will post to a different thread for my actual semi-proposal.  But to
answer the claim that late-bound arg defaults and late-bound
everything-else are unrelated, and to start with an early version of what
I'd actually want, I wrote this:

https://github.com/DavidMertz/peps/blob/master/pep-.rst

I could definitely use help making it better from anyone who is sympathetic
with the idea of generalized deferred computation in Python.

On Tue, Jun 21, 2022 at 5:12 AM Paul Moore  wrote:

> On Tue, 21 Jun 2022 at 10:01, Paul Moore  wrote:
>
> > PS To be clear, my objections to the PEP aren't based on deferred
> > evaluation. So I'm an impartial 3rd party on this matter. I *do* have
> > problems with the PEP, so I have an interest in seeing the PEP fairly
> > reflect the lack of consensus, and accurately represent the concerns
> > people are raising, but I don't have a preference for any specific
> > outcome in the matter of deferred evaluation.
>
> Thinking some more about this, my comments are pretty much what I'd be
> saying if I were a sponsor for this PEP. I don't think a PEP sponsor
> should be someone who doesn't agree with the PEP, otherwise I'd offer
> to take on the role (assuming you need a sponsor). But please take my
> comments in that vein. (And if you do ever manage to convert me to
> support of the PEP, remind me of this comment and I'll be the sponsor
> ;-))
>
> Paul
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/LLEEOLPQTM5INNXRIIT6UVN3TEJLQIZD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BV4MAZPIZA33NMDH7B5W2UFXZ22ZX7AH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Null wildcard in de-structuring to ignore remainder and stop iterating

2022-06-20 Thread David Mertz, Ph.D.

It seems like this is all an occasion to use itertools.tee() ... But with a
consciousness that implicit caching uses memory.

On Mon, Jun 20, 2022, 11:36 PM Steven D'Aprano  wrote:

> On Sun, Jun 19, 2022 at 01:34:35AM +0100, Rob Cliffe via Python-ideas
> wrote:
>
> > To me, the natural implementation of slicing on a non-reusable iterator
> > (such as a generator) would be that you are not allowed to go backwards
> > or even stand still:
> > mygen[42]
> > mygen[42]
> > ValueError: Element 42 of iterator has already been used
>
> How does a generic iterator, including generators, know whether or not
> item 42 has already been seen?
>
> islice for generators is really just a thin wrapper around an iterator
> that operates something vaguely like this:
>
> for i in range(start):
> next(iterator)  # throw the result away
> for i in range(start, end):
> yield next(iterator)
>
> It doesn't need to keep track of the last index seen, it just blindly
> advances through the iterator, with some short-cuts for the sake of
> efficiency.
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/A74YRQJ4QID72GE5I2A3QKOU6NHLJNCD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KC7ATLYOWPGCG4N3A65NJ43ZWNLPR4UE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

On Sun, Jun 19, 2022 at 2:24 PM Chris Angelico  wrote:

> > def frobnicate(data,
> verbose=os.environ.get('LEVEL')==loglevel.DEBUG): ...
>
> Is there any value in not putting that into a global constant?
>

Probably not.  I was just inventing an ad hoc example to show what I
meant.  I didn't search any actual repos I work on for real-life examples.

> Regardless, the @ operator is now available *everywhere* in Python. Does
> it quadratically increase cognitive load?
>

Yeah, probably about that much.  Other than NumPy or closely related array
libraries, I don't know that many other uses. I think I saw something on
PyPI that used it as an email thing, where obviously it has some
familiarity.  But in that case, the lines it occurs on probably have no
more than one or two other sigils.

In the numeric stuff, if I have:

newarray = (A @ B) | (C / D) + (E - F)

That's @, |, /, +, and -.  So 5 operators, and 25 "complexity points".  If
I added one more operator, 36 "complexity points" seems reasonable.  And if
I removed one of those operators, 16 "complexity points" feels about right.

In a function signature "def bisect(stuff, lo=0, hi=None)", you don't
> know what the hi value actually defaults to. Even if it's obvious that
> it is late-bound

Sure, knowing what `hi` defaults to *could be useful*.  I'm sure if I used
that function I would often want to know... and also often just assume the
default is "something sensible."  I just don't think that "could be useful"
as a benefit is nearly as valuable as the cost of a new sigil and a new
semantics adding to the cognitive load of Python.

For example, it also "could be useful" to have syntax that indicated the
(expected) big-O complexity of that function.  But whatever that syntax
was, I really doubt it would be worth the extra complexity in the language
vs. just putting that info in the docstring.

Let's look at a function that has a lot of late-bound default arguments:

pd.read_csv(
filepath_or_buffer: 'FilePath | ReadCsvBuffer[bytes] |
ReadCsvBuffer[str]',
sep=,
delimiter=None,
header='infer',
names=,
index_col=None,
usecols=None,
squeeze=None,
prefix=,
mangle_dupe_cols=True,
dtype: 'DtypeArg | None' = None,
engine: 'CSVEngine | None' = None,
converters=None,
true_values=None,
false_values=None,
skipinitialspace=False,
skiprows=None,
skipfooter=0,
nrows=None,
na_values=None,
keep_default_na=True,
na_filter=True,
verbose=False,
skip_blank_lines=True,
parse_dates=None,
infer_datetime_format=False,
keep_date_col=False,
date_parser=None,
dayfirst=False,
cache_dates=True,
iterator=False,
chunksize=None,
compression: 'CompressionOptions' = 'infer',
thousands=None,
decimal: 'str' = '.',
lineterminator=None,
quotechar='"',
quoting=0,
doublequote=True,
escapechar=None,
comment=None,
encoding=None,
encoding_errors: 'str | None' = 'strict',
dialect=None,
error_bad_lines=None,
warn_bad_lines=None,
on_bad_lines=None,
delim_whitespace=False,
low_memory=True,
memory_map=False,
float_precision=None,
storage_options: 'StorageOptions' = None,

I'd have to look through the implementation, but my guess is that quite a
few of the 25 late-bound defaults require calculations to set that take
more than one line of code.  I really don't WANT to know more than "this
parameter is calculated according to some logic, perhaps complex logic" ...
well, unless I think it pertains to something I genuinely want to
configure, in which case I'll read the docs.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MPDHT2IF5O32AECZKQSYW7AKV6QWGV3X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

>
> > I guess '>=' also looks "confusable", but it's far less common in
> signatures, and the meaning is further away.
>
> It's no less valid than your other examples, nor less common (why
> would you have "==" in a function signature, for instance?).
>

I guess I probably use `==` more often in function calls and signatures, on
reflection.  In call, I use `==` quite often to pass some boolean switch
value, and `>=` much less often.  Obviously, I am aware that `>=` also
produces a boolean result, and YMMV on how often comparing for equality and
inequality expresses the flag you want.

In signature, I'd really only use it, I reckon, as a "default default." E.g.

def frobnicate(data, verbose=os.environ.get('LEVEL')==loglevel.DEBUG):
...

This supposes I have an environmental setting for verbosity that I usually
want to use, but might override that on a particular call.


> > I think the cognitive complexity of a line with sigils is somewhere
> around quadratic or cubic on the number of distinct sigils. But when
> several look similar, it quickly tends toward the higher end. And when
> several have related meanings, it's harder still to read.
>
> It shouldn't need to be. Once you know how expressions are built up, it
> should give linear complexity.
>

I'm not talking about the big-O running time of some particular engine that
parses a BNF grammar here.  I'm talking about actual human brains, which
work differently.

I don't have data for my quadratic and cubic guesses.  Just 40 years of my
own programming experience, and about the same amount of time watching
other programmers.

It would be relatively easy to measure if one wanted to.  But it's a
cognitive psychology experiment.  You need to get a bunch of people in
rooms, and show them lots of code lines.  Then measure error rates and
response times in their answers.  That sort of thing.  The protocol for
this experiment would need to be specified more carefully, of course.  But
it *is* the kind of thing that can be measured in human beings.

So my (very strong) belief is that a human being parsing a line with 5
sigils in it will require MUCH MORE than 25% more effort than parsing a
line with 4 sigils in it.  As in, going from 4 to 5 distinct sigils in the
same line roughly DOUBLES cognitive load.  Here the distinctness is
important; it's not at all hard to read:

a + b + c + d + e + f + g + h + i


And in the weeds, the particular sigils involved (and even the font they're
rendered in) will make a difference too. As well it matters the "semantic
proximity" of the various operators. And other factors too I'm sure.

Did the introduction of the @ (matrix multiplication) operator to
> Python increase the language's complexity multiplicatively, or
> additively? Be honest now: are your programs more confusing to read
> because @ could have been *, because @ could have been +, because @
> could have been ==, etc etc etc, or is it only that @ is one new
> operator, additively with the other operators?
>

I'm not sure how much you know about the background of this in the NumPy
world.  While other libraries have also now used that operator, NumPy was
the driving force.

In the old days, if I wanted to do a matrix multiply, I would either do:

A_matrix = np.matrix(A)
B_matrix = np.matrix(B)

result = A_matrix * B_matrix

Or alternately:

result = np.dot(A, B)


Neither of those approaches are terrible, but in more complex expressions
where the dot product is only part of the expression, indeed `A @ B` reads
better.

And yes, expressions on NumPy arrays will often use a number of those
arithmetic operators I learned in grade school as well as `@`.  But
generally, the mathematics expressed in NumPy code is irreducible
complexity.  It's not necessarily easy to parse visually, but it *IS* the
underlying mathematics.


> > When I write an expression like 'a - b * c / d**e + f' that also has a
> bunch of symbols. But they are symbols that:
> >
> > - look strongly distinct
> > - have meanings familiar from childhood
> > - have strongly different meanings (albeit all related to arithmetic)
>
> The double asterisk wasn't one that I used in my childhood, yet in
> programming, I simply learned it and started using it. What happens is
> that known concepts are made use of to teach others.


I  didn't learn the double asterisk in school either.  That I had to learn
in programming languages.  I actually prefer those programming languages
that use `^` for exponentiation (in that one aspect, not overall more than
Python), because it's more reminiscent of superscript.

Is that simply because you already are familiar with those operators,
> or is there something inherently different about them? Would it really
> be any different?
>

It's a mixture of familiarity and actual visual distinctness.  `/` and `+`
really do just *look different*.  In contrast `:=` and `=` just really look
similar.

Then don't use it in a signature. That's fine. Personally, I've never
> used the "def

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

On Sun, Jun 19, 2022, 12:02 PM Chris Angelico

> I've no idea what you mean by "sophistical" here. Please explain?
>

1.  of or characteristic of sophists or sophistry
2.  clever and plausible, but unsound and tending to mislead
a sophistical argument
3.  using sophistry

Plato's dialog _The Sophist_ is good background.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GH3TETOF4YHWGJ66FTJFNPLFKCUDCQR2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

Unfortunately,  in relation to this PEP, I find your arguments tend to be
sophistical. They ate not generally so in other threads,  but for whatever
reason your attachment to this has a different quality.

There's an interesting point you raise though. You seem to feel that
closely related meanings should have similar looking sigils. I'm not sure
my opinion is *opposite *, but it's definitely more that subtly different
semantics should not be marked by easily visually confusable sigils.

Under PEP 671, a single line of a function signature might contain '=',
':=', '=>', '->', ':', and '=='. Obviously it can have other symbols as
well. But those are the ones the most blur into each other visually. 4 of
them have closely related meanings. You don't even need to be contrived to
have such examples.

I guess '>=' also looks "confusable", but it's far less common in
signatures, and the meaning is further away.

Below you seem to try another unconvincing reductio ad absurdum to suggest
that I must either want a language with no symbols or support this PEP.

I think the cognitive complexity of a line with sigils is somewhere around
quadratic or cubic on the number of distinct sigils. But when several look
similar, it quickly tends toward the higher end. And when several have
related meanings, it's harder still to read.

When I write an expression like 'a - b * c / d**e + f' that also has a
bunch of symbols. But they are symbols that:

- look strongly distinct
- have meanings familiar from childhood
- have strongly different meanings (albeit all related to arithmetic)

Yes, I probably have to think for a while about operator precedence to make
sure I understand it.  Probably I'll add some redundant parens if I can
edit the code. But the cognitive burden of the sigils remains FAR lower
than what would occur with regularity under PEP 671.

I already dislike the walrus operator in signature context for that
reason... although I think it's great for e.g. 'while a := getdata() > 0:
...'

On Sun, Jun 19, 2022, 10:31 AM Chris Angelico  wrote:

> On Mon, 20 Jun 2022 at 00:06, David Mertz, Ph.D. 
> wrote:
> >
> > On Sun, Jun 19, 2022, 3:27 AM Chris Angelico
> >>
> >> > I'm still -1 because I don't think the purpose alone is close to
> worth the cost of new syntax... And especially not using sigils that are
> confusing to read in code.
> >> >
> >> > The topic of "late binding in function signatures"  simply isn't
> *orthogonal* to "late binding in the general sense." Yes, they are
> distinct, but very closely adjacent.
> >>
> >> Every argument you've just made is ALSO an argument against function
> defaults in general. Do you think that they aren't worth syntax either?
> >
> >
> > I honestly can find no connection between what I've argued and "function
> defaults in general." It feels like a non sequitur, but I accept that they
> somehow connect in your mind.
> >
>
> Let's go through the arguments you laid out.
>
> > It's a way to spell `if arg is sentinel: arg = ...` using slightly fewer
> characters, and moving an expression from the body to the signature.
> >
>
> That's what ALL argument defaults do. Some languages evaluate them
> early, some evaluate them late.
>
> > I'm still -1 because I don't think the purpose alone is close to worth
> the cost of new syntax...
> >
>
> What purpose? Being able to define the function's signature in its
> signature, including the meaning of omitted arguments? That's
> definitely the purpose of ALL default arguments, not just late-bound.
>
> > And especially not using sigils that are confusing to read in code.
>
> The sigils used are very similar here: "a=b" and "a=>b". Are you
> really trying to tell me that "=" is not confusing to read in code,
> but "=>" is, or are all such sigils equally confusing? Function
> signatures already have several symbols other than those used for
> defaults, including *, /, and :, not to mention ( ) and -> which
> aren't inside the argument list, but are still part of the function
> signature. Programmers are accustomed to working with symbols (with
> the possible exception of COBOL and DeScribe Macro Language
> programmers), and your argument would work just as well against any
> sort of argument defaults.
>
> > The topic of "late binding in function signatures"  simply isn't
> *orthogonal* to "late binding in the general sense." Yes, they are
> distinct, but very closely adjacent.
>
> This one, you specifically mention late binding, so that's the one
> exception.
>
> Other than that, every one of your arguments could be used to show
> that we shouldn't have ANY fu

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

On Sun, Jun 19, 2022, 3:27 AM Chris Angelico

> > I'm still -1 because I don't think the purpose alone is close to worth
> the cost of new syntax... And especially not using sigils that are
> confusing to read in code.
> >
> > The topic of "late binding in function signatures"  simply isn't
> *orthogonal* to "late binding in the general sense." Yes, they are
> distinct, but very closely adjacent.
>
> Every argument you've just made is ALSO an argument against function
> defaults in general. Do you think that they aren't worth syntax either?
>

I honestly can find no connection between what I've argued and "function
defaults in general." It feels like a non sequitur, but I accept that they
somehow connect in your mind.

Way back last Oct when this discussion had it's first round, someone
(probably Steven, but maybe it was someone else) did a survey of a numerous
programming languages, and whether they use early binding or late binding
of default function arguments.

The result was that MOST languages do late (call time) binding, albeit many
of those are compiled languages where early binding doesn't really make
sense. Some, like JavaScript don't have defaults at all, so the approach is
actually similar to Python:

const myfunc = (a, b, c) => {
if (typeof c === 'undefined') { c = ... }
}

That's a sentinel, but worse than Python in several ways. Others like Perl
don't really have named arguments at all, but that weird implicit list you
can pop from.

The one thing that there were ZERO examples of elsewhere was what you
propose: slightly different syntax to allow both early-bound and
late-bound. Admittedly, if Python had done late-bound from 1.0, probably no
one would now have a PEP to add an early-bound spelling variant.

If Python had always been late-bound, I'm sure I'd be perfectly happy with
it today. But sentinels are really easy, and obscure sigils are really
hard. So adding the new spelling just makes Python worse. Beginners have
something needless to learn, and experienced developers have just a little
more cognitive burden all the time.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4TMLNYHBLQEY3TBWLH3S3DEDCFWN2Q2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

2022-06-18 Thread David Mertz, Ph.D.

On Sat, Jun 18, 2022, 9:21 PM Rob Cliffe

> Sorry again, but IMO discussing any model except one where late-bound
> defaults are evaluated at function call time is just adding FUD.
>

It's definitely rude to repeatedly state that anyone who's opinion is
different from yours is "adding FUD" and doesn't belong in the thread.

Stephen, and Steven, and Paul, and I all perfectly well understand what
"evaluated at function call time" means.

It's a way to spell `if arg is sentinel: arg = ...` using slightly fewer
characters, and moving an expression from the body to the signature.

I won't stoop to saying that advocating what you do is FUD. I can even
understand why someone would want that in Python.

I'm still -1 because I don't think the purpose alone is close to worth the
cost of new syntax... And especially not using sigils that are confusing to
read in code.

The topic of "late binding in function signatures"  simply isn't
*orthogonal* to "late binding in the general sense." Yes, they are
distinct, but very closely adjacent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/INJ6O2MIJUU7ZML2P6SF47FRD7HOMMP6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

2022-06-18 Thread David Mertz, Ph.D.

I think the example that Steven gave, and Stephen approximately repeats is
good.

def func(items=[], n=later len(items)):
items.append("Hello")
print(n)

func()

> I guess Chris will say 0 and David will say 1, but I might be wrong about
either of them.

This is correct. And even though using a (soft) keyword like this gets me
to -0, the semantics I want indeed are different. I only want the binding
to be evaluated when it is referenced. If it never gets referenced, the
compassion time asked side effects are skipped.

Of course, the first line of the function body could be `n = n`. But adding
that line isn't so much different from starting with `if n is None: n = ...
`

As to Stephen's comments on not having used the "evaluate on reference"
pattern, that's pretty much not having used dask.deferred.

The difference is that with Dask (or Haskell) everything stays lazy until
you explicitly call `.compute()` on something in the DAG of operations. I'd
prefer not to need that.

But then my not-a-proposal would need a way to have "a reference that isn't
a reference". I think the same keywords works.

def func(items=[], n=later len(items)):
items.append("Hello")
n = later n**3 # n remains lazy
# ... more stuff
print(n) # actually evaluate the cube of length

On Sat, Jun 18, 2022, 10:52 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> Chris wants syntax for the common pattern
>
> def foo(arg_with_new_empty_list_default=None):
> if arg_with_new_empty_list_default is None:
> arg_with_new_empty_list_default = []
> # do stuff
>
> I can't really guess how useful the "use point" version would be. It's not
> a pattern I've used, I use a zero-argument function very occasionally but I
> can't recall a case where I used a lambda
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RLFYYFG7T53F7XGFGWMTV3NWHWVP3LCU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

2022-06-16 Thread David Mertz, Ph.D.

I've been scolded that I'm not allowed to post unless I support the PEP.
Nonetheless, I reiterate that I oppose it. There is no "preponderance" of
support, but perhaps a slim majority of the small number who have commented
(5 vs 3, I think).

On Thu, Jun 16, 2022, 10:38 PM Steve Jorgensen  wrote:

> Is there anything that I can do, as a random Python user to help move this
> to the next stage? I'm happy to go along with whatever the preponderance of
> responses here seem to think in terms of which syntax choice is best.
> Although I have a slight preference, all of the options seem decent to me.
>
> I am definitely in favor of having the PEP accepted and implemented.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/5572SP7T2GR5PYIVTYN5VESHV5XJ2JA5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/73JW7MNPPHIOCMOFCTD34Z5YMK7ESNS5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

2022-06-15 Thread David Mertz, Ph.D.

As well as all the matters Steven raises, I continue to dislike the
proposal for the same reason I did on earlier rounds.  I believe a general
"deferred computation" mechanism is useful, but that one limited to the
context of function parameters does more harm than good is scoped narrowly
to that single use.  I keyword version might bridge that gap by introducing
"later" or "defer" or "delay" in a narrow context, but not foreclosing its
later use more broadly.

On Wed, Jun 15, 2022 at 8:38 AM Steven D'Aprano  wrote:

> On Tue, Jun 14, 2022 at 11:59:44AM +0100, Rob Cliffe via Python-ideas
> wrote:
>
> > I used to prefer `:=` but coming back to this topic after a long
> > interval I am happy with `=>` and perhaps I even like it more, Chris.
> > The PEP status is "Draft".  What are the chances of something happening
> > any time soon, i.e. the PEP being considered by the Steering Committee?
>
> There's no Sponsor, so it isn't being considered by the SC. That much is
> objectively true.
>
> Beyond that, the following is all my personal opinion, and should not be
> taken as definitive or official in any way. Importantly, I have *not*
> read back through the entire thread to refresh my memory. However, I
> have re-read the PEP in detail.
>
> There's no consensus that this feature is worth the added complexity, or
> even what the semantics are. The PEP punts on the semantics, saying that
> the behaviour may vary across implementations.
>
> There's no consensus on the syntax, which may not matter, the Steering
> Council can make the final decision if necessary. But with at least four
> options in the PEP it would be good to narrow it down a bit. No soft
> keywords have been considered.
>
> In my opinion, there are weaknesses in the PEP:
>
> - lack of any reference to previous discussions;
>
> - no attempt to gather feedback from other forums;
>
> - no review of languages that offer choice of early or late binding;
>
> - little attempt to justify why this is better than the status quo; the
>   PEP seems to take the position that it is self-evident that Python
>   needs this feature, rather than being a balanced document setting out
>   both pros and cons;
>
> - little or no attempt in the PEP to answer objections;
>
> - examples are all chosen to show the feature in the best possible
>   light, rather than to show both the good and bad; (e.g. no examples
>   show the parameter with annotations)
>
> - failure to acknowledge that at least one of the suggested syntaxes
>   is visually ambiguous with existing syntax.
>
> E.g. this would be legal with the PEP's second choice of spelling:
>
> def func(spam, eggs:=(x:=spam)):
>
> Even if the parser can distinguish the two uses of `:=` there, its
> awfully cryptic. In and of itself, that's not necessarily a fatal flaw
> (e.g. slicing) but the benefits have to outweigh the negatives, and the
> PEP should be a balanced discussion of both.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/V5K2JFT44A57ZXV2GS3OS6MQW2YKXMQN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V2WTEHWFOL5LTW76QVGYMAGSNUAHOR6R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Addition to fnmatch.py

2022-06-06 Thread David Mertz, Ph.D.

This is really nice, but I think you have a negation wrong in there
somewhere:

In [1]: from collections import deque
   ...:
   ...: def partition(pred, iterable):
   ...: results = deque([]), deque([])
   ...:
   ...: def gen_split(only):
   ...: for thing in iterable:
   ...: if results[only]:
   ...: yield results[only].popleft()
   ...: results[not pred(thing)].append(thing)
   ...: for thing in results[only]:
   ...: yield thing
   ...:
   ...: return gen_split(True), gen_split(False)
   ...:

In [2]: def isEven(n):
   ...: return n/2 == n//2
   ...:

In [3]: from itertools import count

In [4]: evens, odds = partition(isEven, count())

In [5]: next(evens)
Out[5]: 1

In [6]: next(evens)
Out[6]: 3

In [7]: next(evens)
Out[7]: 5

In [8]: next(odds)
Out[8]: 0

In [9]: next(odds)
Out[9]: 2

In [10]: next(odds)
Out[10]: 4

On Mon, Jun 6, 2022 at 10:32 AM Mathew Elman  wrote:

> from collections import deque
>
> def partition(pred, iterable):
> results = deque([]), deque([])
>
> def gen_split(only):
> for thing in iterable:
> if results[only]:
> yield results[only].popleft()
> results[not pred(thing)].append(thing)
> for thing in results[only]:
> yield thing
>
> return gen_split(True), gen_split(False)
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/EUAYPXMZM4DOGTAAUURNOMWTDU65K6UY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HWKJMLZSQVZQERYAKYZUAMK2YGIFUPIX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Addition to fnmatch.py

2022-06-06 Thread David Mertz, Ph.D.

As Chris Angelico notes, that's only good for finite iterables.

The recipe in the itertools docs is unbounded, but it's also unbounded in
the cache needed if you get long strings of things the do (or don't)
confirm the predicate.

On Mon, Jun 6, 2022 at 7:06 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> Christopher Barker writes:
>
>  > How do you divide a sequence into two sequences cleanly?
>
> I don't know how to do it in a one liner, but
>
> yes = []
> no = []
> for elem in seq:
> (yes if test(elem) else no).append(elem)
>
> seems pretty clean to me.
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/BBVBDT7KJTTOD5VQYIHBCQBU5QV32ZUB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4RIL6JUJXJDF3G6HWVU4DD33TKPEZYCI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add .except() and .only(), and .values_at(). instance methods to dict

2022-06-05 Thread David Mertz, Ph.D.

These are all far too easy to do with comprehensions to merit new methods
or stdlib functions.

m = {'a': 123, 'b': 456, 'c': 789}
>
m.except(('a', 'c'))  # {'b': 456}
> m.only(('b', 'c'))  # {'b': 456, 'c': 789}
> m.values_at(('a', 'b'))  # [123, 456]
>

{k:m[v] for k in m if k not in ['a','c']}
{k:m[v] for k in m if k in ('b','c')}.
[m[k] for k in m if k in {'a', 'b'}]


The existing versions are a few characters longer, but far more general
when you want minor variations.  Moreover, I have very rarely wanted to do
ANY of the things described by these hypothetical methods.

That said, I think maybe something extending PEP 584 could be reasonable,
but I'm not quite sure of the semantics.

E.g., we might provide additional set-like operators for dicts.

>>> m | {'a':"Newval"}  # We have this now
{'a': 'Newval', 'b': 456, 'c': 789}

>>> m - {'a'}  # Would rightval be a set or dict though?!
{'b': 456, 'c': 789}

>>> m & {'a', 'b'}  # Same question, but set feels better, I think
{'a': 'Newval', 'b': 456}


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B23J2ABBFOOXQYHYSVHX7ILJZRRE2IKH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Addition to fnmatch.py

2022-06-05 Thread David Mertz, Ph.D.

On Sun, Jun 5, 2022, 12:08 PM MRAB  wrote:

> On 2022-06-05 16:12, David Mertz, Ph.D. wrote:
> > This is exactly the problem solved by set difference. E.g. `{a, b, c}
> - {a, c}`.
> >
> > This operation is linear on the size of the removed set.
> >
> You don't have to use a set difference, which might matter if the order
> was important, but just make a set of the ones found:
>
>  s = set(b)
>
> And then the list comprehension:
>
>  [x for x in a if x not in s]
>

Sure, that's nice enough code and has the same big-O complexity. I suspect
set difference is a little faster (by a constant multiple) because it hits
C code more, but I haven't benchmarked.

The OP said the elements were from fnmatch though, which explicitly does
not promise order. So really it's just whether you like your code or this
better aesthetically:

list(set(b) - set(a))

Personally I like my (somewhat shorter) version as more clear. But YMMV.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FD3JEDTKB4COFBRSLGIT53Y2GFW6MTZF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Addition to fnmatch.py

2022-06-05 Thread David Mertz, Ph.D.

This is exactly the problem solved by set difference. E.g. `{a, b, c} - {a,
c}`.

This operation is linear on the size of the removed set.

On Sun, Jun 5, 2022, 11:01 AM John Carter  wrote:

> I’d like to propose a simple addition to the 'fnmatch' module. Specificity
> a function that returns a list of names that match a pattern AND a list of
> those that don't.
>
> In a recent project I found that I wished to split  a list of files and
> move those that matched a pattern to one folder and those that didn't to
> another folder. I was using fnmatch.filter to select the first set of files
> and then a list comprehension to generate the second set.
>
> For a small number of files (~ 10) this was perfectly adequate. However as
> I needed to process many files (>>1) the execution time was very
> significant. Profiling the code showed that the time was spent in
> generating the second set. I tried a number of solutions including
> designing a negative filter, walking the file system to find those files
> that had not been moved and using more and more convoluted ways to improve
> the second selection. Eventually I gave in and hacked a local copy of
> fnmatch.py as below:
>
> def split(names, pat):
> """Return the subset of the list NAMES that match PAT."""
> """Also returns those names not in NAMES"""
> result = []
> notresult = []
> pat = os.path.normcase(pat)
> pattern_match = _compile_pattern(pat)
> if os.path is posixpath:
> # normcase on posix is NOP. Optimize it away from the loop.
> for name in names:
> if not pattern_match(name):
> result.append(name)
> else:
> notresult.append(name)
> else:
> for name in names:
> if not pattern_match(os.path.normcase(name)):
> result.append(name)
> else:
> notresult.append(name)
> return result, notresult
>
> The change is the addition of else clauses to the if not pattermath
> statements. This solved the problem and benchmarking showed that it only
> took a very small additional time (20ms for a million strings) to generate
> both lists
>
> Number of tests cases 100
> Example data ['Ba1txmKkiC', 'KlJx.f_AGj', 'Umwbw._Wa9', '4YlgA5LVpI’]
> Search for '*A*'
> TestTime(sec)   PositiveNegative
> WCmatch.filter  1.953125   26211  0
> filter 0.32812514259  0
> split  0.34375014259  85741
> List Comp.   270.468751 14259  85741
>
> The list comprehension [x for x in a if x not in b]*, was nearly 900 times
> slower.
>
> ‘fnmatch’ was an appropriate solution to this problem as typing ‘glob’
> style search patterns was easier than having to enter regular expressions
> when prompted by my code.
>
> I would like to propose that split, even though it is very simple, be
> included in the 'fnmatch' module.
>
> John
>
> *a is the original and b is those that match.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/EZEGFGJOHVHATKDBJ2SWZML62JWT2VE2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/72KFWQJHIFUJJRH32WIGEZ64BRN7UK6E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Delete dictionary entry if key exists using -= operator via isub()

2022-04-28 Thread David Mertz, Ph.D.

Using `dct -= key` is WAAAYY too cryptic. A method would be okay, but since
that is just `dct.pop(key, None)` without the second argument, it's not
worth the trouble.

-1

On Thu, Apr 28, 2022, 12:03 PM Zach Victor  wrote:

> Good points, Antoine!
>
> The intent is to mutate the dictionary with no side effects if a key
> exists — and without the extra code that comes along with a control
> structure, a try except, or positing a None to throw away.
>
> For me, pop() says "give me something." Since the intent is to mutate the
> dictionary only and not "get something," I'd prefer (i) not to use a
> function that returns a value and (ii) not to specify a return value None
> just so that it can be thrown away. Perhaps if pop()'s default return value
> were None—as it is with, say, get()—I would use pop() as you describe. But
> pop() doesn't have a default value unless you give it one, and pop() always
> returns a value.
>
> Where does this land with PEP 20? I think the use of pop() as you suggest
> lands on the implicit side of things and is not as readable: the reader has
> to ask, "what are we doing with the default value? Oh. Nothing. It's to
> delete a dict entry." However, pop() with the default value of None is
> practical, and practicality does beat purity.
>
> I agree that operator overloading needs to be consistent and easy to
> understand, and in that regard -= with the dict key probably does not
> withstand the PEP 20 tests. Furthermore, in support of your point, the
> language has moved in the direction of set operations between dicts using
> operators: for example, + for union of dicts. To satisfy set- and key-based
> operations, one could overload the augmented arithmetic assignment
> operators so that -= with a string would delete the entry with a matching
> key, while -= with a dict would delete the entry with a matching key and
> value. Again, that may be too recherché or hard-to-explain (cf PEP 20).
>
> Perhaps, then, it would be best to leave dict alone and just overload
> operators in subclasses for specific uses!
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ICUUGL7DABI3VTL4GDSNVOM7D3HEO467/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZKRCW2ZPKKZW6TLV7SHLK4CJRJPBDB3N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditions for a coherent MI relationship [was Re: Re: mro and super don't feel so pythonic]

2022-04-16 Thread David Mertz, Ph.D.

R has changed a bit since 2005. My article was from then:
https://gnosis.cx/publish/programming/R3.html

I'm not trying to quibble about R, but simple to point out that what often
mystified as "deep" about OOP is actually banal and fairly trivial.

That might be how Alan Kay originally saw OOP. He famously regretted using
> the term "object" because it distracted from what he saw as the genuinely
> fundamental parts of OOP, namely
>
> * Message passing
> * Encapsulation
> * Late (dynamic) binding


I like Alan. I worked with him a little bit. I agree with his attitude here.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SXHA3GUDZ7P5EO6RF3YB5LGGCOHE5XXV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditions for a coherent MI relationship [was Re: Re: mro and super don't feel so pythonic]

2022-04-16 Thread David Mertz, Ph.D.

On Sat, Apr 16, 2022, 12:26 PM Chris Angelico > wrote:

> A button IS a widget.
> A horizontal box IS a box.
> A box IS a container.
> A container IS a widget.
>
> Class hierarchies in graphical systems are just as much based on those
> is-a relationships as any of those "horrible metaphors" you may have come
> across, and they are absolutely to do with programming. Just because you
> had a bad teacher, that doesn't mean the concepts are wrong.
>

Well, more like a few dozen bad books. I never studied any of this with a
teacher. But when I teach it (to quite a lot of programming professionals),
I never once mention that IS-A stuff, other than maybe curses sotto voce.
And mostly they come away saying they finally understood it after my
presentation.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5A3J6LRBADRYWC65MXHIQNHPQICLDDIA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditions for a coherent MI relationship [was Re: Re: mro and super don't feel so pythonic]

2022-04-16 Thread David Mertz, Ph.D.

I don't really care what terms like "Full MI" or "True MI" are intended to
mean. Python and MANY other programming languages that allow classes to
have multiple parents, use a method resolution order (MRO) to decide which
same-named method from a complex inheritance tree to use in a given call.

If languages use an MRO, C3 is probably the best choice. But not the only
one. But sure, C++ and Eiffel are free to refuse "ambiguity" of names
occurring multiple times in an inheritance tree. Other languages are free
to use single inheritance, or traits, or mixins.

I wrote an article a lot of years ago, around the time C3 was new to
Python, intending to show what a banal thing OOP is in which I added
inheritance and an MRO to R (all within R itself). It was something like
20-30 lines of code in total. My MRO was something dumb like depth first or
breadth first, but it WAS an MRO. R doesn't have inheritance, it's not OOP,
which was my main reason to choose it for the exercise.

One thing I do find a bête noire is the silly claim, that Chris repeats,
that inheritance expresses "Is-A" relationships. It was torture in the
1990s hearing about vehicles and trucks and sedans as if that had something
to do with programming. I never "got" OOP for at least 5 years later than I
should have because of those horrible Linnaean metaphors.

The reality is more clear in the actual primary definition of the word
itself: "the practice of receiving private property, titles, debts,
entitlements, privileges, rights, and obligations". In programming, a class
can receive methods and attributes from some other classes. That's all.
It's just a convenience of code organization, nothing ontological.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/CSSBRRM42NK7JMLMOEBL7JIY5ITPQC2W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-04-12 Thread David Mertz, Ph.D.

On Tue, Apr 12, 2022, 4:31 PM Steven D'Aprano

> Except the most important one: why are you using multiple inheritence when
> you clearly don't want multiple inheritence, instead you want delegation?
>

I'll note that hundreds of posts ago, I made the FIRST reply to this
silliness, and that it 100% explained every single needless digression that
followed:

> If you want to explicitly delegate a method to a class, you should
explicitly delegate a method to a class.
> This is exactly why a lot of folks feel composition is better than
inheritance (at least often so).
> You don't need `super()` to call `SomeSpecificClass.method(self, other,
args)`

I also co-authored a widely read article on then-new C3 linearization in
Python with Michele Simianato before he wrote his really great explanation
of it in the Python docs, and his also great 15+ year old pieces on traits
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T3TAQ46HJ2LATLPYEP36LRLEZEA5YYCH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-04-10 Thread David Mertz, Ph.D.

Can you think of ANY context in Python in which the order of items in
parentheses isn't important?!

(a, b, c) != (c, b, a)

func(a, b, c) != func(c, b, a)

etc.

You are arguing that defining inheritance order is "intuitively" the one
and only context in which the order of items in parentheses makes no
difference.



On Sun, Apr 10, 2022, 1:23 PM malmiteria  wrote:

> David Mertz, Ph.D. writes:
> > Are you likewise "confused" by the fact that `a = b - c` is generally
> > different from `a = c - b`?!
>
> Why do you always quote confused?
>
> I'm not, but that's because i've been taught / i've experienced that since
> primary school. I have been taught math, but not python, nor any programing
> language.
>
> Most people experience with python will lead them to understand easily
> that parenthesis are definitely not symetrical, since calling a function is
> a common thing to do in python, and most programming language.
> As much as they would be able to understand that there is many different
> context in which what's define inside the parenthesis doesn't always mean
> the same thing. And overall, the "parenthesis operation" doesn't exists as
> one simple thing.
> Sure there's __call__, but parenthesis are also used in method definition,
> method call and class definition, on top of class calls.
> As much as there is some importance of the order of the arguments in
> method definition, it's definitely a fair assumption, based on generic
> experiences with the language, that parenthesis in the syntax "class A(B)"
> simply means something different, and is a different operation.
>
> However, the class definition allows you to refer the class by the name
> placed before the parenthesis, so it's obvious even only with a generic
> python experience that the "inside/outside" parenthesis order matter.
> Much more than the order within the parenthesis.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/43AEEIGIBPFEPKAIZWPN34PCVGR4QWBD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V4VEYL4OJBYAJZLMI4DKWWWPSWNEMSZM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-04-10 Thread David Mertz, Ph.D.

On Sun, Apr 10, 2022, 11:51 AM malmiteria

> "class A(B)" feels very different from "class B(A)"
> but "class A(B,C)" doesn't feel so obviously different from "class A(C,B)"
>

Are you likewise "confused" by the fact that `a = b - c` is generally
different from `a = c - b`?!

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7DC66QOXIKRCXM7EJ522HUMCIDIVCVK5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-04-10 Thread David Mertz, Ph.D.

On Sun, Apr 10, 2022, 10:35 AM Steven D'Aprano

> I don't believe that David's denials that people are confused by super()
> are even a little bit reasonable.


I take the words "unconcerned" and "confused" to be distinct.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IPIWPFOHRYJ2S6KKJ5C6P52JOLKPARFN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-04-09 Thread David Mertz, Ph.D.

On Sat, Apr 9, 2022, 7:31 AM malmiteria  wrote:

> Joao S. O. Bueno writes:
> > You are still repeating this:
> > "more in line with the expectation of the majority, "
> > Though, as already asked, there is zero (nothing) to support that.
>

Here's some more evidence of a sort: I've taught hundreds, maybe thousands,
of scientists and software developers Python. Some only knew different
programming languages, some wanted to understand Python more deeply.

Malmiteria is the FIRST person I've met "confused" by super().

Most of those are not confused because they've simply never had any reason
to give it deeper thought. It never did and never will matter to them. A
much smaller number were not confused because they are accustomed to using
deep and branched inheritance trees, and therefore read Michele's paper on
C3 MRO.

Another open question is the threshold at which we would all agree there's
> a problem to be fixed i guess?
>

I guess the main threshold to try to cross is ONE person other than
malmiteria in the universe of Python users.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HFQKMUNA6MQMYJF3KESXN227H7J34X22/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Giving Decimal a global context was a mistake?

2022-04-07 Thread David Mertz, Ph.D.

On Thu, Apr 7, 2022 at 2:47 PM Marc-Andre Lemburg  wrote:

> In high finance, I've never seen decimals being used, only floats.
> Excel is omnipresent, sets the standards and uses IEEE 754 floats
> as well (plus some black magic which sometimes helps, but often
> makes things worse):
>

In forex, instantaneous exchange rates are defined as a specific number of
decimal digits in a currencyA/currencyB exchange rate (on a particular
market).

This is about US$6.6 trillion/day governed by these rules... FAR more than
the combined size of ALL securities markets.

It was something like 2007 when the NYSE moved from prices in 1/32 penny to
a fixed-length decimal representation.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KD7YAXR4CU5SRD3Q67PIG7S6VMFWVMOV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Giving Decimal a global context was a mistake?

2022-04-07 Thread David Mertz, Ph.D.

Because the module implements http://speleotrove.com/decimal/decarith.html

On Thu, Apr 7, 2022 at 2:03 AM Greg Ewing 
wrote:

> On 7/04/22 9:53 am, Chris Angelico wrote:
> > how else would you do variable-precision in the standard library?
>
> Maybe the mistake was in thinking that we need variable
> precision at all.
>
> If the goal of Decimal was to provide arithmetic that
> "works like your calculator", well, most calculators have
> a fixed precision of 10 or so digits, which seems to be
> fine for most things.
>
> So why *do* we have variable precision in Decimal?
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/OECXPUSG2SD3YEVEQ4OXCP7NODOYJ3XR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MYLDUL4XCFZLYBUXXX7MIKZWMA5SQ2EI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

2022-04-07 Thread David Mertz, Ph.D.

On Thu, Apr 7, 2022 at 10:51 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> NOTE: The big problem I see with this is that I don't see any
> practical way to use syntax for non-numeric types like ndarray.
> The problem is that in, say, economics we use a lot of m x 2
> arrays where the two columns have different units (ie, a quantity
> unit and a $/quantity unit).  The sensible way to express this
> isn't some kind of appended syntax as with numbers, but rather a
> sequence of units corresponding to the sequence of columns.
>

Here's a few minutes of work by me:

>>> df = pd.DataFrame([['Apples', 11, 2], ['Pears', 12, 3]],
...   columns=[Units("Item", "Kind-of-Fruit"), Units("Per-Bushel",
"USD/bushel"), Units("Number")])
...
...
>>> df
 Item  Per-Bushel  Number
0  Apples  11   2
1   Pears  12   3
>>> df.columns[1].units
'USD/bushel'


I did this by defining:

>>> class Units(str):
... def __new__(self, s, units="Dimensionless"):
... return str.__new__(self, s)
... def __init__(self, s, units="Dimensionless"):
... float.__init__(s)
... self.units = units


If this were actually needed, I'm sure better interfaces could be created.
But this very simple thing is a possible way to retain units per-column.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5GC5PZJJDHZRMIG7X46GDUCQOQSM5I5H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

On Mon, Apr 4, 2022 at 7:45 PM Ethan Furman  wrote:

> On 4/4/22 13:31, David Mertz, Ph.D. wrote:
>
>  > You should probably change the thread subject to "All-and-only 7 SI
> units" if that's what you mean.
>
> While I'm sure SI would be very useful, I suspect that any system will
> have to be useful for a much broader audience to
> be accepted; given the vast amount of units, I suspect individual
> libraries will be needed for each area/discipline.
>

That's pretty much exactly my point.  Every time anyone points out
complications or difficulties, Brian falls back to "we only care about the
7 SI units, all other uses are illegitimate."  Since that is his position,
he should state that in the thread.

I think a LIBRARY (not syntax) that handles the real complication of units
is an excellent thing. A couple do exists, but I'm sure they could be
further improved.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZNTUKEJBWDFWOLI47Q5BXD3FV7HVL3VR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

You should probably change the thread subject to "All-and-only 7 SI units"
if that's what you mean.

On Mon, Apr 4, 2022 at 1:46 PM Brian McCall 
wrote:

> > I drove 20 minutes up the road to by a bushel (US, not British) of U.S.
> No. 1. apples, to make apple cider.  On my return trip, I stopped at the
> hardware store to buy a 2 lb box of 1-3/4" ring shank 12 penny nails.  I
> used my 7/8 hole kitchen planer blade to grate the apples, then squeezed
> them for an hour and a 15 minutes at 30 psi to extract the juice.  For good
> measure I added 2 tablespoons of vanilla and a pinch of salt.  Then I drove
> the nails into grade C 2x4 joists (whose sizes are 1.5 x 3.5 inches, with a
> 1/16th inch permissible tolerance in sizing).
>
> I'm going to start by ignoring any quantities above that are not involved
> in any sort of calculation. Those are outside the scope of the problem and
> proposal. And I don't care what anyone thinks of that.
>
> The only calculation I see here is 1.5 in x 3.5 in = 0.00339 mm2. The fact
> that joists are called 2x4 even though their nominal dimension is 1.5 in x
> 3.5 in is also outside the scope of the problem and proposal. Come on man,
> you think you're the only one who knows examples like this? How about 1/3"
> image sensors? How about display diagonals? I know about these things, and
> I know that they do not need to be accounted for in a standard,
> language-supported representation of units.
>
> Units have infinite precision, so grades and tolerances are also
> irrelevant.
>
> The units you mentioned are:
> minutes
> bushel (imperial)
> bushel (US)
> psi
> tablespoon
> pinch (it's a stretch, but okay)
> foot
> inch
>
> 7/8 hole - this is a specification, not a unit
> 2x4 joist - specification, not a unit
> grade C - specification, not a unit
> 12 penny - Wikipedia calls it a unit, but calculations in measurements
> taken in units of pennies are neither associative nor distributive, and
> transformations on measurements taken in units of pennies are neither
> additive nor multiplicative.
>
> Anyway, you mentioned you knew of at least 1000 units. I count 7. You have
> another 993?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/DHRG6STCIHJ2XOF6SE7O27UG7MX66DS5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LQPY7AJZ5P6LVPN27NCBL4SUICOHVPNS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

>
> Units have infinite precision, so grades and tolerances are also
> irrelevant.
>

Not if you believe in Planck lengths (or quantum states) :-).

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2CSWETFQQBXRVBBPGBFTKTOPKPHG2TE3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

Just to elaborate on units I use, here's a hypothetical (not stuff that
actually happened today, but very commonplace nature):

I drove 20 minutes up the road to by a bushel (US, not British) of U.S. No.
> 1. apples, to make apple cider.  On my return trip, I stopped at the
> hardware store to buy a 2 lb box of 1-3/4" ring shank 12 penny nails.  I
> used my 7/8 hole kitchen planer blade to grate the apples, then squeezed
> them for an hour and a 15 minutes at 30 psi to extract the juice.  For good
> measure I added 2 tablespoons of vanilla and a pinch of salt.  Then I drove
> the nails into grade C 2x4 joists (whose sizes are 1.5 x 3.5 inches, with a
> 1/16th inch permissible tolerance in sizing).


Please express that description in SI units! ;-)

On Mon, Apr 4, 2022 at 11:55 AM Ken Kundert  wrote:

> I think there is one more point worth making here.  There is a suggestion
> that
> dimensional analysis can underpin a units system.  Actually, the idea that
> all
> units can be broken down into a small set of fundamental units is very
> limiting
> and results in many vexing issues.
>
> For example, consider currencies.  There are currently hundreds of
> national
> currencies and thousands of cryptocurrencies.  They all have the same
> basic
> fundamental unit of “value”, but value is only loosely defined.
> Furthermore,
> there is no fixed ratio between the currency and its value.  It varies
> over
> time, over location, and from person to person.
>
> Consider units of a particular commodity.  The example of a ream of paper
> was
> recently mentioned. A ream is 500 sheets of paper.  However two reams may
> not be
> comparable.  They may refer to a different size of paper or a different
> quality
> of paper.  So all prices for reams of paper would have the same
> fundamental
> units of “value per each”, but both “value” and “each” are not necessarily
> comparable.  In effect, the fundamental unit system is not complete.  You
> also
> need to include information about what you are measuring.  For example,
> “each”
> could represent a single item of anything.  The unit is not complete until
> you
> include a description of what that anything is, and in effect, there is an
> unlimited number of things it could be.
>
> Now consider the issue of “unitless units”.  In electrical circuit we
> often talk
> about gain, which it the ratio between the signal level at the output of
> a circuit relative to the signal level at the input.  But you need to be
> specific about how you measure signal level.  Typically, it would be a
> voltage,
> current, or power level.  Consider a circuit where both the input and
> output are
> measured in volts.  Then the gain would have units of "V/V", which is
> unitless
> in dimensional analysis.  But units of "V/V" (voltage gain) is much
> different
> from units of "A/A" (current gain) or "W/W" (power gain), even though they
> have
> the same dimensions.  Mixing them up results in errors and confusion.  An
> additional complication is that sometimes logarithmic units are used.  For
> example, decibels in voltage, or dBV, is 20*log(Vout/Vin).  Again,
> a dimensionless quantity, but nonetheless "dBV" much different from "V/V".
>
> The same issue occurs with the arguments to trigonometric functions like
> sin(),
> cos() and tan().  Generally, we assume the arguments are given in radians,
> which
> is a dimensionless number.  But it could also be given in degrees, another
> dimensionless number.  Radians and degrees are indistinguishable from the
> perspective on dimensional analysis, but mixing them up results in errors.
>
> This is not to knock the idea of dimensional analysis.  It is just not
> something
> that would be done in most programs that process physical quantities.
> Rather it
> is something that is largely done as a one-time check on your analysis.
> It is
> a “second opinion” on whether your hand calculation are correct, or at
> least
> plausible.  So dimensional analysis packages such as pint have their
> place, but
> dimensional analysis is not something that belongs in the base language or
> even
> the standard library.
>
> However I do believe that a case can be made to allow numbers to be easily
> tagged with units in the base language, and then allowing those units to
> be
> accessed as an attribute of the number.  Packages such as pint and
> QuantiPhy
> could then use that attribute to provide processing of units that is
> appropriate
> for the particular application.
>
> -Ken
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/A6BVGOUYE2BPO54M63KXFNDOE6FUSJ3F/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the

[Python-ideas] Re: Native support for units [was: custom literals]

On Mon, Apr 4, 2022 at 3:53 AM Chris Angelico wrote:

> The 160 units would be more likely to have collisions though. Also,
> the base and derived SI units will be used with magnitude prefixes,
>

The supposed 160 are far fewer than I use on a daily (or at least weekly)
basis. Yes, all the rest are convertible, but that's not how they're
actually used. A hectare or acre represents the square of a unit of
distance (and a constant). But no one (realtors or land surveyors goes
through the large inconvenience of converting units, constants, and
multipliers, when they work with the units.

And no one baking a cake cares that you can cube a distance to get a (dry)
volume of flour.

Reams are a "unitless" constant of 500, but no one ever uses them for
things other than paper... plus that fact that they used to be 480, so the
unitless number itself has an implicit date attached to it.

Yes... if you ignore the real world, and real users of Python, units can be
made so simple as to potentially be syntax.

In the real world, and as I stated, 160 is at least an order of magnitude
too low.

--
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons. Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/7BWJJTUHY3SKYCOH2CCYVKUXR2HQRTXA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

On Mon, Apr 4, 2022, 2:17 AM Brian McCall
> For instance, I don't think there would be much uproar if "teaspoons"
were left out of any kind of implementation.

Apparently someone other than you does the cooking in you household!
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UOU2CSY3OM6VTUE5OEBUK5G6EWGK75EB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

2022-04-03 Thread David Mertz, Ph.D.

On Mon, Apr 4, 2022, 12:53 AM Brian McCall

> > An electron volt is a unit of energy. Or of mass. Or of momentum.
> An electron volt is a unit of energy and only a unit of energy. Knowing a
> particle's energy (in certain situations) means that you also know other
> physical quantities about that object, and so in casual conversation (and
> the occasional poorly reviewed journal article) you find them used
> interchangeably.


This is just flatly wrong of usage in particle physics. Electron volts are
precisely the default units used to describe the mass of subatomic
particles.

Would it help if we stopped saying "units" and instead referred to
> "standard units"?


Yes, limiting the idea to ""SI units" would cover far less. And thereby
have far less motivation to change Python syntax rather than use a library.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/55MZRW6JAQPHYNWMPDGAXKN475WA5CQG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Native support for units [was: custom literals]

2022-04-03 Thread David Mertz, Ph.D.

While units are certainly useful, I think this is FAR too large to add to
Python syntax. A library like "units" is great, but it really needs to be a
library, and not a small one.

There are thousands of units in use in sciences, commerce, engineering,
ordinary life, etc. In all of them, it is not uncommon for the same few
letters to represent multiple things, depending on context.

But as well as the sheer number, what is convertible to what, and in what
context, is very nuanced.

An electron volt is a unit of energy. Or of mass. Or of momentum. It's
usual to convert among these meanings for these units. But bordering on
nonsense to convert among kilowatt-hours and grams. It depends on the
practical context of units.

Even in Ethan's example, he uses kilogram as a unit of weight. It answers a
question with the word weight in it (albeit 300 kg is extreme for a human
weight, maybe for brick pallets). But kg IS NOT a unit of weight. It's a
unit of mass. In the context of thing on the surface of the earth near the
same elevation, we might convert between kg and lbs. But in other contexts
it makes no sense.

I'm older than 35 (Earth) years, but my age is also counted differently
according to different calendars and cultural conventions.  Of his
examples, °F is the only one that is completely convertible to other
temperature scales, but even there, it's rare to see a bread recipe in
degrees Kelvin.

Units like time intervals and currencies are EXCEEDINGLY complicated. The
conversion rate between Dirham and USD fluctuates by microsecond, but can
also have two simultaneous values on different exchanges. At the least, a
unit of currency has built into it an absolute time, but also often a
relative time to a specific refence time.

... So then we get to the point of saying "don't do any of that, just
annotate the number with an arbitrary string/object." Which sure, is
possible. Not even difficult. But we can also do it already without syntax
by subclassing int, or float, or complex, or Decimal, or Fraction. Or we
can do it just with name conventions too.  Perhaps 'person_kg' is just a
numeric variable reminding us of its unit.

On Sun, Apr 3, 2022, 6:25 PM Chris Angelico  wrote:

> On Mon, 4 Apr 2022 at 07:45, Ethan Furman  wrote:
> > Mechanically, are `lbs`, `km`, `hr`, etc., something that is imported,
> or are they tags attached to the numbers?  If
> > attached to the numbers, memory size would increase and performance
> might decrease -- but, how often do we have a number
> > that is truly without a unit?
> >
> > How old are you?  35 years
> > How much do you weigh?  300 kg
> > What temperature do you cook bread at?  350 F
> >
>
> Very frequently - it'd be called an index. (What sort of numbers
> should enumerate() return, for instance? Clearly that, whatever it is,
> is an index.) But if every int and float has a tag attached to it,
> it's not that big a deal to have either a default tag, or leave the
> field NULL or None, to define it to be a unitless value or index.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/7LTKMLQFJHQYU2TGQCPYBXXX6472M3H7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JO2IBCSMVUOX34IL3NJWWPF64RLBYVFY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

2022-03-26 Thread David Mertz, Ph.D.

If you want to explicitly delegate a method to a class, you should
explicitly delegate a method to a class.

This is exactly why a lot of folks feel composition is better than
inheritance (at least often so).

You don't need `super()` to call `SomeSpecificClass.method(self, other,
args)`

On Sat, Mar 26, 2022, 12:59 PM malmiteria  wrote:

> Hi,
>
> Before anything, i made a github repository about this topic here :
> https://github.com/malmiteria/super-alternative-to-super
>
> The core of what i wanna discuss here is that i don't think mro and super
> (mainly because it relies on mro) are very pythonic. Mainly that some
> behaviors of the mro are too implicit, and are silencing what really should
> be errors.
>
>
>
> Let me explain :
> in case of multiple inheritence, resolving a child method from it's parent
> isn't an obvious task, and mro comes as a solution to that. However, i
> don't understand why we don't let the programmer solve it. I think this is
> similar to a merge conflict, and not letting the programmer resolve the
> conflict feels like silencing an error. This is especially infuriating when
> you realise that mro doesn't solve all possible scenarios, and then, simply
> refuses the opportunity to solve it to the programmer.
> Then, super relying on mro gives off some weird behaviors, mainly, it's
> possible for a child definition to affect what a call to super means in
> it's parent. This feels like a side effect (which is the 'too implicit'
> thing i refer to).
> I also don't understand why we can't simply pass the parent targeted as
> argument to super, instead of having no argument, or having to pass the
> current class and instances as argument :
> super(child) is a proxy to parent, when super(parent) would make more
> sense to be the proxy to parent, in my mind.
>
> I dive in more depths about those topics in the readme of the github
> repository i linked at the top of this comment.
>
>
>
> what i propose is a solution that would follow those rules:
>
> The mro alternative, which i called explicit method resolution aka EMR
> (which is probably not a good name since i apply it, as mro, to all class
> attributes), follow those rules :
>  1) Straightforward case : the class definition has the method / attribute
> : this is the one EMR should resolve to
>  2) Not found : the method / attribute can't be resolved in the class
> itself, or by any of it's parents, then it should raise an AttributeError
>  3) Only on parent : the method / attribute can't be resolved in the class
> itself, and can only be resolved by one of it's parents, this is the one
> EMR should resolve to
>  4) Multiple parent : the method / attribute can't be resolved in the
> class itself, and can be resolved by at least 2 of it's parents, then an
> ExplicitResolutionRequired error should be raised
>  5) Transimittin errors : the method / attribute can't be resolved in the
> class itself, and one parent at least raises an ExplicitResolutionRequired
> error, then it should raise an ExplicitResolutionRequired error
>  6) (optional?) Single source : when multiple parent can resolve a method
> from a single source (in case of diamond shape inheritence), the
> ExplicitResolutionRequired  is not needed
>
> The super alternative, which i called __as_parent__ should follow those
> rules :
>  1) reliability : the target __as_parent__ points to should not depend on
> anything other than the argument passed to it
>  2) expliciteness : in case of multiple inheritence, the parent targetted
> should be passed as an argument to the __as_parent__ method.
>  3) impliciteness : in case of simple inheritence, it is not needed to
> specify the parent targeted (since there can only be one, and it make it
> closer to the actual behavior of super in most cases)
>  4) ancestors as targets : should be able to target ancestors, either
> direct or not (which is needed in case two grandparent define a method that
> a single parent share, there would be no other way to solve the
> ExplicitResolutionRequired otherwise)
>
>
>
> this solution has a few advantages in my mind :
>  - the current mro and super are more tightly coupled than the emr and
> __as_parent__ i propose here
>  - the __as_parent__ is more reliable than super in its behavior, and
> should lead to an easier learning curve
>  - the emr i propose as a replacement to mro allows for some inheritence
> tree mro doesn't allow.
>  - the __as_parent__ method being able to target specific parent allows
> for different methods to visit the parents in different order easily, which
> today would be harder, since the parent visiting order is tightly coupled
> to the class definition
>  - with emr, in case of problematic resolution, an error is raised to tell
> you about the problem, and ask you for explicit resolution, the current
> solution doesn't, which can lead to surprises closer to production
> environment.
>
>
>
> A few possible downsides :
>  - the transition would be a pain to deal with
>

[Python-ideas] Re: A modulo operator consistent with euclidean division.

2022-03-18 Thread David Mertz, Ph.D.

Are you speaking of how to generalize modulo to negative numbers (about
which programming languages vary)?

The bar for adding new syntax to Python is VERY high. Probably easier is
getting a function in the `math` module. But even there, you'll need to
explain what behavior you want AND why that behavior is sufficiently
commonly wanted to warrant inclusion in the stdlib.

What are the use cases you have in mind?

On Fri, Mar 18, 2022, 11:10 AM Nathan Levett 
wrote:

> Howdy python gang,
>
> First time posting here ~ I've recently encountered that python does not
> have an OOTB operator for modulo that is consistent with Euclidean
> division. Although it should be easy for anyone who wants this to create it
> themselves, it struck me as odd that it was not an already included
> feature. I was even more shocked to see a list indicating that most
> languages don't include one consister with Euclidean division
> (disheartening to realise that the number theory way of doing things is not
> the default). I was so shocked at it's lack from python that it motivated
> me to post this, I suppose!
>
> I guess I'm posting to check how open anyone would be to the idea? I'm not
> sure if '%%' is defined anywhere, but it seemed like an intuitive
> suggestion if not already used as an operator, parallel to the syntax of
> the '**' operator.
>
> Keen to know how open y'all're to it!
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/KRISFMIOXQOXFH5Y4XYUMBEONMCCOOQH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NUYNVA27PR72FGNTOEQGHTGEKD4QF22F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: An unambiguous way of initializing an empty set and dictionary

2022-03-17 Thread David Mertz, Ph.D.

I just do this myself in my text editor (vim):
[image: sets-py.png]

But this is just cosmetic because I like to look at it this way.  The
actual file on disk contains `set()`, `<=`, `in`, `not in` and wouldn't be
a problem for anyone without the same fonts installed, or require anyone to
know odd key combos.


On Thu, Mar 17, 2022 at 8:35 AM Michael Smith  wrote:

> Hmm, I think the idea of the mathematical symbol is interesting, but I
> think users are more interested in constructing a new, eventually-not-empty
> set, than referencing the empty set.
>
> Semantically, I don't know if ∅() is satisfying.
>
> On Thu, Mar 17, 2022 at 08:19 Stéfane Fermigier  wrote:
>
>> The “correct” (according to Bourbaki) mathematical notation for an empty
>> set is “∅" (aka Unicode U+2205, or HTML )
>>
>> Some time ago, for a project which had a lot of empty sets, I tried to
>> use this symbol as a short hand for set(). But:
>>
>> >>> ⦰ = set()
>>   File "", line 1
>> ⦰ = set()
>> ^
>> SyntaxError: invalid character '⦰' (U+29B0)
>> >>> ø = set()
>> >>>
>>
>> In other words, “⦰” is illegal as an identifier in Python (same for  ⌀
>> aka U+2300 DIAMETER SIGN), but “ø” (aka U+00F8 LATIN SMALL LETTER O WITH
>> STROKE) is legal !
>>
>> So I used  "⌀" instead of “⦰”, but I eventually dropped the whole idea
>> because, IIRC, some tools weren’t too happy with it.
>>
>> Still, I guess it wouldn’t be neither too hard nor two disruptive to
>> accept “⦰” as well as some other mathematical characters as identifiers in
>> Python.
>>
>> Since once of the application domains where Python shines nowadays is
>> mathematics (numerical, but also symbolic), I think it’s a shame that we
>> are preventing to use the proper unicode characters to designate some
>> universal mathematical objects.
>>
>> More info here: https://en.wikipedia.org/wiki/Null_sign
>>
>>   S.
>>
>> On 13 Mar 2022 at 22:52:16, joao.p.f.batista...@gmail.com wrote:
>>
>>> Currently:
>>> l = [] # new empty list
>>> t = () # new empty tuple
>>> s = set() # new empty set (no clean and consistent way of initializing
>>> regarding the others) <<<
>>> d = {} # new empty dictionary
>>>
>>> Possible solution:
>>> s = {} # new empty set
>>> d = {:} # new empty dictionary (the ":" is a reference to key-value
>>> pairs)
>>>
>>> Current workaround at least for consistency:
>>> l = list() # new empty list
>>> t = tuple() # new empty tuple
>>> s = set() # new empty set
>>> d = dict() # new empty dictionary
>>>
>>> However, it doesn't feel right to not be able to initialize an empty set
>>> as cleanly and consistently as lists, tuples and dictionaries in both forms.
>>> ___
>>> Python-ideas mailing list -- python-ideas@python.org
>>> To unsubscribe send an email to python-ideas-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-ideas@python.org/message/OPWEKLLGT6RWMEHC4SLEZ2LSRVCSS4GP/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>> --
>> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier
>> - http://linkedin.com/in/sfermigier
>> Founder & CEO, Abilian - Enterprise Social Software -
>> http://www.abilian.com/
>> Co-Founder & Co-Chairman, National Council for Free & Open Source
>> Software (CNLL) - http://cnll.fr/
>> Co-Founder & Chairman, Association Professionnelle Européenne du Logiciel
>> Libre (APELL) - https://www.apell.info/
>> Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA)
>> - https://www.euclidia.eu/
>> Founder, PyParis & PyData Paris - http://pyparis.org/ & http://pydata.fr/
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/D3SSAHRXYHPAM4NW3E6XC3J3AL625PGL/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Z43FW7AEEVFXJZ5CVLAPPD2A2PCMLOMY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to

[Python-ideas] Re: Add a replace method to tuples

2022-03-14 Thread David Mertz, Ph.D.

On Mon, Mar 14, 2022, 7:52 PM Steven D'Aprano

> I got "No results matched your query" on the second URL, at which point I
> didn't bother with the first.
>

Hmm... while I don't support the proposal, I saw results at both links.
Those links were the first good faith comments I saw from OP.

A pretty large number of prominent projects use either:

foo = bar[:n] + newval + bar[n+1:]

Or:

newfoo = list(foo)
newfoo[n] = newval
foo = tuple(newfoo)

Somewhere in their codebases. I find that mild evidence in favor of the
idea. But the pattern seems to be "occasionally in large-ish projects."

On the other hand, looking through those examples in a cursory manner made
me think about 75% of the time "why are they doing this silly approach?"
and 25% "this is easy enough as is." None of them made me think "I wish I
had a method/function."
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2FZ6N2W4FUK3IA4OICA2UZASQTZ6FPIO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: An unambiguous way of initializing an empty set and dictionary

2022-03-14 Thread David Mertz, Ph.D.

On Mon, Mar 14, 2022 at 8:33 AM  wrote:

> Possible solution:
> s = {} # new empty set
> d = {:} # new empty dictionary (the ":" is a reference to key-value pairs)
>

I have suggested over the years—as have probably dozens of other people
(maybe thousands)—that that would be a great spelling if Python were a
brand new language.  But in reality, Python is 30+ years old, and billions
of lines of code use `{}` as an empty dict.

Historically, Python had dictionaries before it had sets, so there was more
than a decade in there where sets were not a thing you could spell at all.
A breaking change isn't something that's going to happen here.

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7U4MOQ4O46MMJO6OA34ZGML7X2WRVFR3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a replace method to tuples

2022-03-14 Thread David Mertz, Ph.D.

On Fri, Mar 11, 2022, 6:19 PM wfdc  wrote:

> > Likewise this which you wrote in another post: "Do you see why it's
> useful to have immutability?"
>
> That's not insulting at all. It's a perfectly valid question to address to
> a post that seems to be contesting the purpose of immutable types in the
> first place.
>

Gosh, I had no idea immutability could be useful! :-)

https://openlibra.com/en/book/functional-programming-in-python
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3TOPNZCWORHCR6CY272RRTBITFU54IFR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a replace method to tuples

Yes. To be clear, immutability isn't a use case, it's a particular
technique that can be useful for solving some problems.

This is probably a clear enough example of the XY-problem as to be worth
adding to the Wikipedia article on that topic.

We sometimes see other similar proposals to e.g. "solve" the "problem" that
Python isn't statically typed. That is, folks who want Python to be some
different language.


On Fri, Mar 11, 2022, 4:42 PM wfdc  wrote:

> (To be clear, I'm saying David Mertz's proposal fails the immutability
> criterion, not Christopher Barker's.)
>
> --- Original Message ---
> On Friday, March 11th, 2022 at 4:39 PM, wfdc  wrote:
>
> Don't yell.
>
> You just effectively re-implemented Christopher Barker's solution (which
> was also present in the StackOverflow thread), with the downside that it
> fails the immutability criterion.
>
> Saying "just be careful not to mutate the original datastructure" isn't a
> solution. There's a reason we have immutable types: To *enforce*
> immutability. Otherwise, why aren't you proposing getting rid of the tuple
> type entirely?
>
> --- Original Message ---
> On Friday, March 11th, 2022 at 4:29 PM, David Mertz, Ph.D. <
> david.me...@gmail.com> wrote:
>
> On Fri, Mar 11, 2022, 4:16 PM wfdc via Python-ideas <
> python-ideas@python.org> wrote:
>
>> > why haven't you used a list
>> 2. I don't want to modify the original sequence.
>>
>
> There's a really easy solution for you that will even be more perfomant.
>
> Use a list and DON'T modify the original!
>
> This is ABSOLUTELY an XY-problem which fact was difficult to wrestle
> out of you.
>
> >>> stuff1 = [a, b, c, d]
> >>> stuff2 = stuff1[:]
> >>> stuff2[2] = e
>
>
>
>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I5NRWYBXQSJ4SYQY7HFMWLVN4YUTRJY6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a replace method to tuples

So it appears the "problem" this is intended to solve is "Python isn't
Haskell."

But Haskell exists, and this is a non-problem looking for a solution.


On Fri, Mar 11, 2022, 4:39 PM wfdc  wrote:

> Don't yell.
>
> You just effectively re-implemented Christopher Barker's solution (which
> was also present in the StackOverflow thread), with the downside that it
> fails the immutability criterion.
>
> Saying "just be careful not to mutate the original datastructure" isn't a
> solution. There's a reason we have immutable types: To *enforce*
> immutability. Otherwise, why aren't you proposing getting rid of the tuple
> type entirely?
>
> --- Original Message ---
> On Friday, March 11th, 2022 at 4:29 PM, David Mertz, Ph.D. <
> david.me...@gmail.com> wrote:
>
> On Fri, Mar 11, 2022, 4:16 PM wfdc via Python-ideas <
> python-ideas@python.org> wrote:
>
>> > why haven't you used a list
>> 2. I don't want to modify the original sequence.
>>
>
> There's a really easy solution for you that will even be more perfomant.
>
> Use a list and DON'T modify the original!
>
> This is ABSOLUTELY an XY-problem which fact was difficult to wrestle
> out of you.
>
> >>> stuff1 = [a, b, c, d]
> >>> stuff2 = stuff1[:]
> >>> stuff2[2] = e
>
>
>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q2UKBLSDXZ3DVTPHHCINA7XPLMF3XCEH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a replace method to tuples

On Fri, Mar 11, 2022, 4:16 PM wfdc via Python-ideas 
wrote:

> > why haven't you used a list
> 2. I don't want to modify the original sequence.
>

There's a really easy solution for you that will even be more perfomant.

Use a list and DON'T modify the original!

This is ABSOLUTELY an XY-problem which fact was difficult to wrestle
out of you.

>>> stuff1 = [a, b, c, d]
>>> stuff2 = stuff1[:]
>>> stuff2[2] = e
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LZAWS3HDMJOVWJYZW2OIKSFSB5D4VILY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a replace method to tuples