[Python-ideas] Re: heapq max heap

[Nick Timkovich]

I myself have proposed heap objects and seen that and other features come
up from time to time. I think the general mood is to use a PyPI library, or
even just vendor a single file from a package like xheap
https://github.com/srkunze/xheap/blob/master/xheap.py

If you wanted to stick with what's in the stdlib and have a minimal
wrapper, you could turn your heap inputs into tuples of form
`(key_function(object), object)`, and on getting them out, just get the
object back. To have a max heap, key_function could just be operator.neg.

On Wed, Apr 13, 2022 at 6:25 PM Eduardo Nery 
wrote:

> Many times when using the heapq lib I had to convert my array to negative
> values so that it could behave like a max heap. Maybe we should pass in a
> parameter in the functions to specify if we want a max heap behaviour or a
> min heap behaviour.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/S5EF5VZKNGJCYA7NQM5ESUZJC4DXXDV4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PL7AJ4NRBQ3PXCNQPDBCC276QJMB3N7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Creating ranges with ellipsis

[Nick Timkovich]

>
> > This might be a silly idea but, would it be a good idea to have
> > ...[a:b:c] return a range(a, b, c)?
>

If a 'thunderscore' is acceptable:

import itertools
class _ranger:
@classmethod
def __getitem__(self, key: slice):
if isinstance(key, slice):
if key.stop is None:
return itertools.count(key.start, key.step or 1)
return range(key.start, key.stop, key.step or 1)
return range(key)
___ = _ranger()

Trying to write it brings out lots of questions like what would [:y] do, or
[:], [::z], etc. Only [x], [x:y], [x:], [x::z], [x:y:z] seem to make sense.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WARMRS7GMQHYEBGR5FTBKHW436DWRWW6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Regex timeouts

[Nick Timkovich]

>
> A regex that's vulnerable to pathological behavior is a DoS attack waiting
> to happen. Especially when used for parsing log data (which might contain
> untrusted data). If possible, we should make it harder for people to shoot
> themselves in the feet.
>

While definitely not as bad and not as likely as SQL injection, I think the
possibility of regex DoS is totally missing in the stdlib re docs. Should
there be something added there about if you need to put user input into an
expression, best practice is to re.escape it?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ODUC75DKJTFWSD227S7S2W6HVUV5JCZ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Issue 34850: Syntax Warning in the real world (is 0 / is 1)

[Nick Timkovich]

>
> I understand it locks in that particular implementation detail, but it
> also seems unlikely that that particular optimization (int 0 and int 1 as
> specific memory objects) would ever make sense to NOT be including in a
> shipping implementation (future proof by practicality).
>

Do you explicitly want to differentiate between *CPython integer* 0's and
1's and floats? How about Numpy and other numeric libraries?

assert 0.0 is not 0
assert 1.0 is not 1
assert numpy.ones(1, dtype=int)[0] is not 1
assert numpy.int8(1) is not 1

I think you're going to cause sneaky bugs for yourself.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HLXJHCBM27BWZFWWZRBRBLO2EFYOK7EJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add Standard String Literals and Prefixes for Mathematical Notation

[Nick Timkovich]

Can you provide short, but non-trivial, clear examples of "before" (current
Python) and "after" (what you propose it looks like) to demonstrate the
advantage?

Will it be ambiguous with existing syntax?

On Tue, Feb 25, 2020 at 6:10 PM Nathan Edwards 
wrote:

> I love regular expressions. I would love to see Bra-Ket notation and many
> of the popular mathematical forms commonly practiced in engineering and
> science supported by the Python language in an expressive and logical way.
> I feel the need for expressing mathematical concepts in a standardized and
> sightly way, beyond just being able to use Unicode equivalents of
> mathematical symbols, would be an enormous functional benefit.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HDES3QWONVQSSK6JRKDHMB5RPU4MD42C/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C7MOBMWHKOLQRSKBAYJXWVGTJPU6CKI4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Control adding script path/CWD to sys.path

[Nick Timkovich]

I guess I'm confused if you want (e.g.) `python json.py` to do something
but not `python -m json` to call the same file (a different way), given
you're in the directory.

nick@lappy386:~/Code$ echo "{}" | python -m json.tool
{}
nick@lappy386:~/Code$ touch json.py
nick@lappy386:~/Code$ echo "{}" | python -m json.tool
.../bin/python: Error while finding module specification for 'json.tool'
(ModuleNotFoundError: __path__ attribute not found on 'json' while trying
to find 'json.tool')
nick@lappy386:~/Code$ echo "{}" | python -I -m json.tool
{}

Isolated mode effectively ignores the environment (variables and
directory), so it will only use what's in that Python's site-packages
(non-user, but could be a virtualenv)

If you want the local path to not be on `sys.path` for a *script-like*
call, you could do `python -I myscript.py`, but `-m mypkg` is I think
indistinguishable from trying to `import mypkg`, so it needs to be
discoverable on `sys.path`...this feels like an A/B problem: what are you
trying to do?

On Mon, Feb 24, 2020 at 3:35 PM  wrote:

> Nick Timkovich wrote:
>
> > > Are you familiar with the -I option for "isolated mode"?
> > https://docs.python.org/3/using/cmdline.html#id2
> > -I
> > Run Python in isolated mode. This also implies -E and -s. In isolated
> mode
> > sys.path contains neither the script’s directory nor the user’s
> > site-packages directory. All PYTHON* environment variables are ignored,
> > too. Further restrictions may be imposed to prevent the user from
> injecting
> > malicious code. New in version 3.4.
>
> Oooh... Can you provide some examples on the use of -I?
>
> I try to use along with -m (`python -I -m a.b`) and get this error:
> "python: Error while finding module specification for 'a.b'
> (ModuleNotFoundError: No module named 'a')".
>
> On the other hand `python -m a.b -I` just passed -I as a script's CLI
> argument.
>
> Of course, I get the expected results if run `python -I` on terminal.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/QTZRM5QZI6DI7D43XVYX2GR75IX5SUFX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IDG3CKB6MRIG2T4I5LEAK5SMUHRBMNRT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Control adding script path/CWD to sys.path

[Nick Timkovich]

On Mon, Feb 24, 2020 at 11:51 AM Brianvanderburg2 via Python-ideas <
python-ideas@python.org> wrote:

> When running a python script directly, the directory of that script gets
> added to sys.path.  When running as a module "python -m", it looks like an
> empty string gets added to sys.path, which appears to result in CWD being
> used.
>

Are you familiar with the -I option for "isolated mode"?
https://docs.python.org/3/using/cmdline.html#id2

-I
Run Python in isolated mode. This also implies -E and -s. In isolated mode
sys.path contains neither the script’s directory nor the user’s
site-packages directory. All PYTHON* environment variables are ignored,
too. Further restrictions may be imposed to prevent the user from injecting
malicious code. New in version 3.4.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NXGL4FT37MZ6QEPYQBJVI6Y2K4O2WMWF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Traits

[Nick Timkovich]

On Fri, Feb 7, 2020 at 10:11 AM Soni L.  wrote:

> I'd like to see traits some day, with a syntax similar to this one:
> ...
> if the trait isn't used in the function definition you get the raw
> object, where name conflicts between traits (but not between traits and
> inherent methods) result in an error about name conflicts. otherwise you
> get a friendly wrapper.
>

I assume traits are a feature of another language, but not being familiar
with it can you illustrate its need a bit better? Can you give an example
in current Python, and how it could be made more clear with the notional
trait syntax?

Nick
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H5F6WOALTZ6IYUTVS7QVEI4KZG2TREMN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add boolean attribute to datetime objects stating if they are tz-aware

[Nick Timkovich]

Currently the datetime docs say:

> An object of type time or datetime may be naive or aware. A datetime
object d is aware if d.tzinfo is not None and d.tzinfo.utcoffset(d) does
not return None. If d.tzinfo is None, or if d.tzinfo is not None but
d.tzinfo.utcoffset(d) returns None, d is naive. A time object t is aware if
t.tzinfo is not None and t.tzinfo.utcoffset(None) does not return None.
Otherwise, t is naive.

> https://docs.python.org/3/library/datetime.html#available-types

Would an issue/PR be welcome to encode that logic into code? The docs could
then read:

> An object of type time or datetime may be naive or aware. Aware objects
have their "aware/tzaware/tz_aware" attribute set to True, and if naive:
False.

Nick
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4G5L5YW4V7X2APBVK4DBTT22IDKIZXME/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Distutils] Re: Linux binary wheels?

[Nick Timkovich]

On Tue, Aug 20, 2019, at 5:05 AM Matthew Brett 
wrote:

> ...  Unless you meant wheels for non-Intel platforms, in which case,
> please do say more about you need.


Minor tangent: I've seen some people use https://www.piwheels.org/ for
Raspberry Pi (ARM 6/7), but could the ARM binaries be uploaded to PyPI?

I think I'm conflating the wheel building spec (is manylinux amd64
specific, or as long as the libraries are on any architecture?),
toolchains, environment (sounds like Piwheels provides a platform to build
them on), and package hosting (can PyPI host arbitrary archs?) in that one
sentence.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OXSUW73EO5DTUO34EFURN3KHCDAKNS4Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP: Idiomatic API for shell scripting

[Nick Timkovich]

I actually gave a talk along these lines at the Chicago Python (ChiPy)
meetup last week: slides
https://docs.google.com/presentation/d/1v5z4f-FQkS-bQYE-Xv5SvA6cyaTiqlxs2w2CI1yZcAU/edit?usp=sharing

Part of the argument was about using pure standard library so a
self-contained script/repo could run anywhere Python is in order to (e.g.)
bootstrap other testing environments and/or work within restricted ones,
just like your average shell script. A gigantic step up from there is using
*anything* not in the stdlib because you may need to copy a bunch of files
(venv creation), and download executable things from *the Internet*
(horrific to some).

On Sat, Aug 10, 2019 at 11:03 PM David Mertz  wrote:

> In various different ways, also:
>
> https://pypi.org/project/bash/
>
> https://pypi.org/project/python-pipe/
>
> https://pypi.org/project/pypette/
>
> https://pypi.org/project/shell/
>
> Just a few I've seen before. Xonsh is pretty cool also.
>
>
> On Sat, Aug 10, 2019, 9:55 PM Christopher Barker 
> wrote:
>
>> On Sat, Aug 10, 2019 at 6:53 AM David Mertz  wrote:
>>
>>> Something very similar to this had been done many times in third party
>>> libraries.
>>>
>>
>> Check out Xonch for example:
>>
>> https://xon.sh/
>>
>> -CHB
>>
>>
>>
>>
>>> None of those have become hugely popular, including none quite
>>> compelling me personally to do more than try them out as toys.
>>>
>>> It's too bad, in a way, since I love bash and use it all the time when
>>> it seems easier than Python.
>>>
>>> Your library is 100% certainly not going to become part of the standard
>>> library unless it can first become popular as a third party tool. If it
>>> does become popular that way, I wouldn't put it's odds of eventual soon to
>>> the standard library as higher than 5-10%. But that doesn't really matter,
>>> widespread use his 'pip install' world be a good thing in itself.
>>>
>>> On Sat, Aug 10, 2019, 7:52 AM Thiago Padilha 
>>> wrote:
>>>
 Python is often used a shell scripting language due to it being more
 robust than traditional Unix shells for managing complex programs, but
 it is significantly more verbose than Bash & friends for doing common
 shell scripting tasks such as writing pipelines and redirection .

 I propose adding an idiomatic shell scripting API that uses Python
 operator overloading in a way that allows invoking commands in a
 Bash-like way.

 A fully working implementation (which I've been using in my personal
 scripts) can be found at https://github.com/tarruda/python-ush/. Here
 I will describe the basics of the API, more details in the github
 README.rst (which is also a doctest for the project).

 Here's how one can create commands:

 >>> from shell import Shell
 >>> sh = Shell()
 >>> cat = sh('cat')
 >>> ls = sh('ls')
 >>> sort = sh('sort')
 # or something like this:
 >>> cat, ls, sort = sh('cat', 'ls', 'sort')

 The Shell class is the entry point of the API and it is used as a
 factory for Command objects. For convenience we can add a builtin
 Shell instance which is a also module from where commands can be
 quickly imported from (idea taken from sh.py
 http://amoffat.github.io/sh/):

 >>> from shell.sh import cat, ls, sort

 We can construct more complex commands by calling it with certain
 arguments:

 >>> ls('-l', '-a', env={'LANG': 'C.UTF-8'})

 And we can invoke the command by calling it without arguments:

 >>> ls()
 # or
 >>> ls('-l', '-a', env={'LANG': 'C.UTF-8'})()

 Invoking the command returns a tuple with the status code:

 >>> ls()
 (0,)

 The are more ways to invoke a command other than calling it without
 arguments. One such example is by iterating through it, which
 automatically takes care of piping the output back to python:

 >>> files = []
 >>> for line in ls:
 ... files.append(line)

 Pipelines can be easily created with the `|` operator:

 >>> ls = ls('--hide=*.txt', env={'LANG': 'C.UTF-8'})
 >>> sort = sort('--reverse')
 >>> ls | sort
 ls --hide=*.txt (env={'LANG': 'C.UTF-8'}) | sort --reverse

 Everything that can be done with Command objects, can also be done
 with Pipeline objects:

 >>> (ls | sort)()
 (0, 0)  # single commands return a tuple to make the return value
 compatible with Pipelines
 >>> list(ls | sort)
 [u'tests', u'setup.cfg', u'pytest.ini', u'bin', u'README.rst']

 We also use the Pipeline syntax for redirecting input/output.

 # piping to a filename string redirects output to the file
 >>> (ls | sort | '.stdout')()
 (0, 0)
 >>> str(cat('.stdout'))
 'tests\nsetup.cfg\npytest.ini\nbin\nREADME.rst\n'
 # piping from a filename 

[Python-ideas] Re: Shouldn't this: ' myVar: "value" ' be a syntax error ?

[Nick Timkovich]

On Tue, Jul 9, 2019 at 4:15 PM Andrew Barnert via Python-ideas <
python-ideas@python.org> wrote:

> On Jul 9, 2019, at 13:09, Shay Cohen  wrote:
> >
> > >>> a: 3
>
> The reason this isn’t a syntax error is that Python allows any valid
> expression as an annotation. And “3” is just as valid an expression as
> “int”.
>
> More generally, annotations don’t actually do anything at runtime, except
> get stored in an annotation dictionary.
>

If you're curious where those annotation "definitions" actually disappear
off to: `__annotations__`

>>> a: 3
>>> __annotations__
{'a': 3}
>>> class X:
... b: 42
...
>>> hasattr(X, 'b')
False
>>> X.__annotations__
{'b': 42}
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U7EJB3K5UQSOA6DM3OH6RAGN3LFPCVZR/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Built-in parsing library

[Nick Timkovich]

What does it mean to be a universal parser? In my mind, to be universal you
should be able to parse anything, so you'd need something as versatile as
any Turing language, so one could stick with the one we already have
(Python). I'm vaguely aware of levels of grammar (regular, context-free?,
etc.), and how things like XML can't/shouldn't be parsed with regex [1].
Most protocols probably aren't *completely* free to do whatever and
probably fit into some level of the hierarchy, what level would this
putative parser perform at?

Doing something like this from-scratch is a very tall order, are there
candidate libraries that you'd want to see included in the stdlib? There is
an argument for trying to "promote" a library that would security into the
standard library over others that would just add features: trying to make
the "one obvious way to do it" also the safe way. However, all things
equal, more used libraries tend to be more secure. I think suggestions of
this form need to pose a library that a) exists, b) is well used and
regarded, c) stable (once in the the stdlib things are hard to change), and
d) has maintainers that are amenable to inclusion.

Nick

[1]: https://stackoverflow.com/a/1732454/194586

On Sat, Mar 30, 2019 at 12:57 PM Nam Nguyen  wrote:

> Hello list,
>
> What do you think of a universal parsing library in the stdlib mainly for
> use by other libraries in the stdlib?
>
> Through out the years we have had many issues with protocol parsing. Some
> have even introduced security bugs. The main cause of these issues is the
> use of simple regular expressions.
>
> Having a universal parsing library in the stdlib would help cut down these
> issues. Such a library should be minimal yet encompassing, and whole parse
> trees should be entirely expressible in code. I am thinking of combinatoric
> parsing as the main candidate that fits this bill.
>
> What do you say?
>
> Thanks!
> Nam
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Left arrow and right arrow operators

[Nick Timkovich]

In general, there is lots of code out in the wild that can't be updated for
whatever reason, e.g. the person that knows Python left and it needs to
continue to work. Weak argument, but cost-benefit I think it comes out
ahead. In your example there isn't a reason I can tell why swapping the
operands isn't what should be done as Calvin mentioned. The onus is on you
to positively demonstrate you require both directions, not him to
negatively demonstrate it's never required.

I suggest you confine your proposal to `->` only, as it's currently illegal
syntax. You would also want the reflected `__r*__` equivalent of
`__arrow__` or `__rarrow__` (`__rrarrow__` if you also need the
left-arrow...)

Perhaps broadening the use of it, functions may be able to use it as a pipe
operator, e.g. Elixir:
https://elixir-lang.org/getting-started/enumerables-and-streams.html#the-pipe-operator

On Mon, Mar 11, 2019 at 2:58 PM francismb  wrote:

> Hi Greg,
>
> On 3/9/19 1:42 AM, Greg Ewing wrote:
> > Do you really want
> > to tell them that all their code is now wrong?
> Of course not, at least not so promptly. But, would it be still a
> problem if the update to a new version (let say from 3.X to next(3.X))
> is done through some kind of updater/re-writer/evolver. In that case the
> evolver could just add the blanks. What do you think ? Could it work?
>
> Thanks in advance!
> --francis
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using sha512 instead of md5 on python.org/downloads

[Nick Timkovich]

Devils advocate: it might complicate things for someone that needs to use
FIPS, where MD5 can be a pain to deal with.

On Fri, Dec 7, 2018 at 8:50 AM Devin Jeanpierre 
wrote:

> On Fri, Dec 7, 2018 at 1:40 AM Antoine Pitrou  wrote:
>
>> md5 is only used for a quick integrity check here (think of it as a
>> sophisticated checksum).  For security you need to verify the
>> corresponding GPG signature.
>>
>
> More to the point: you're getting the hash from the same place as the
> binary. If one is vulnerable to modifications by attackers, both are. So it
> doesn't matter. The real defense most people are relying on is TLS.
>
> -- Devin
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposing additions to the standard library

[Nick Timkovich]

Not to derail the conversation, but I've always been curious why the
itertools recipes are recipes and not ready-made goods (pre-baked?) that I
can just consume. They're great examples to draw from, but that shouldn't
preclude them from also being in the stdlib.

On Mon, Nov 12, 2018 at 7:41 PM Michael Selik  wrote:

>
>
> On Sat, Nov 10, 2018 at 6:56 PM Jonathan Crall  wrote:
>
>> Sometimes there's a good, useful function than doesn't get added because
>>> there's no reasonable place to put it. For example, a "flatten" function
>>> has been talked about since Python 1.x days, and we still don't have a
>>> standard solution for it, because (1) it isn't clear *precisely* what it
>>> should do, and (2) it isn't clear where it should go.
>>
>>
>> The flatten example is good to know about. Is there a link to this
>> discussion or a summary of it? I would think flatten could go in itertools,
>> but clearly there must some reason why its not there. I imagine the
>> duplication with it.chain.from_iter + "There should be one-- and preferably
>> only one --obvious way to do it."?
>>
>
> https://docs.python.org/3/library/itertools.html#itertools-recipes
> There's an example of ``flatten`` in the itertools recipes.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add "default" kwarg to list.pop()

[Nick Timkovich]

Does it make sense to draw some sort of parallel between next(myiterator,
default="whatever") and mylist.pop(default="whatever")? They exhaust the
iterator/list then start emitting the default argument (if provided).

On Thu, Nov 1, 2018 at 2:46 PM Serhiy Storchaka  wrote:

> 31.10.18 13:08, Antoine Pitrou пише:
> > +1 from me.  dict.pop() already has an optional default.  This is a
> > straight-forward improvement to the API and no Python programmer will
> > be surprised.
>
> list.pop() corresponds two dict methods. With argument it corresponds
> dict.pop(). But there are differences: dict.pop() called repeatedly with
> the same key will raise an error (or return the default), while
> list.pop() will likely return other item. Without argument it
> corresponds dict.popitem() which doesn't have an optional default.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Powerset

[Nick Timkovich]

"more-itertools" seems to be a modestly popular library (top 100-500?) that
includes the stdlib itertools recipes and more.
https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.powerset

On Tue, Oct 16, 2018 at 3:42 AM Hasan Diwan  wrote:

> Pal,
> On Tue, 16 Oct 2018 at 01:36, Pål Grønås Drange 
> wrote:
> > I do however agree that there could be a powerset function there for
> convenience, but only +0.
>
> That is the best argument I could come up with to justify a
> set#powerset method. -- H
> --
> OpenPGP:
> https://sks-keyservers.net/pks/lookup?op=get=0xFEBAD7FFD041BBA1
> If you wish to request my time, please do so using
> bit.ly/hd1AppointmentRequest.
> Si vous voudrais faire connnaisance, allez a bit.ly/hd1AppointmentRequest.
>
> Sent from my mobile device
> Envoye de mon portable
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] On evaluating features [was: Unpacking iterables for augmented assignment]

[Nick Timkovich]

On Tue, Aug 28, 2018 at 12:57 PM Guido van Rossum  wrote:

> However, a user who doesn't typically think about the actual semantics of
> iterable unpacking and tuple packing might think this would instead mean
> the following:
>
>   a += x
>   b += x
>   c += x
>
> IOW they might think that this is a clever way to increment three
> variables at once:
>
>   a, b, c += 1
>
> This ambiguity (in naive users' minds) leads me to frown upon the proposal.
>

This sort of broadcasting is often quite convenient in Numpy, where many
arithmetic operators will accept scalars or vectors and generally "do the
right thing" in the context. It gets complicated in higher dimensions, but
for 0/1-dimensions you don't need an axis-selector.

abc = np.array([1, 2, 3])
abc += 1
print(abc)
# [2 3 4]

abc += [10, 100, 1000]
print(abc)
# [  12  103 1004]

The quirks I see are with + being a concatenation operator for lists/tuples.

a, b = (10, 20)
# a, b += (1, 2)
# a, b == 11, 22?
(a, b) + (1, 2)
# (10, 20, 1, 2)

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Idea: msgfmt.py and pygettext..py should be -m executable modules

[Nick Timkovich]

On Mon, Jul 30, 2018 at 8:43 AM, Petr Viktorin  wrote:

> On Mon, Jul 23, 2018 at 2:16 PM, Miro Hrončok  wrote:
>
> > It might be:
> >
> >   $ python3 -m gettext
>
> +1
>
> > And:
> >
> >   $ python3 -m gettext.msgfmt
>
> +1
> Note that this means gettext will need to become a package.
>
> > And (provided as a shortcut):
> >
> >   $ python3 -m msgfmt
>
> -1. This would mean adding a new top-level module to the standard
> library. Let's not pollute that namespace just to provide a shortcut.


Speaking of shortcuts, is there a reason that it should be "python -m
json.tool" versus "python -m json" (keep the old probably forever, but have
the shorter as a synonym)?

And as a broader question, is there opposition to generating patches to
make packages executable that may be useful as a CLI? Would be nice if the
UUID or random modules could directly spit something out, e.g. "python -m
uuid -4". Counterargument that you could "python -c "import
uuid;print(uuid.uuid4())".

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a __cite__ method for scientific packages

[Nick Timkovich]

On Fri, Jun 29, 2018 at 8:58 PM, Matt Arcidy  wrote:

> It seems like the very small percentage of academic users whose careers
> depend on this cannot resolve the political issue of forming a standards
> body.
>
> I don't see how externalizing the standard development will help.  Kudos
> for shortcutting the process in a practical way to just get it done,  but
> this just puts core devs in the middle of silly academic spats.  A language
> endorsed citation method isn't a 'correct' method, and without the broad
> consensus that currently doesn't exist, this becomes _your_ method, a
> picked winner but ultimately a lightning rod for bored tenured professors
> with personal axes to grind.  If this were about implementing an existing
> correct method I'm sure a grad student would be tasked with it for an
> afternoon.
>
>
[...]  Just create a jstor style git server where obeying the citation
> protocol is mandatory.
>

I don't know if it constitutes a standards body, but there are a couple
journals out there that are meant to serve as mechanisms for turning a repo
into a published/citable thing, they might be good to look at for prior art
as well as to what metadata should be included:

* https://joss.theoj.org/about (sponsored by NumFOCUS)
* https://www.journals.elsevier.com/softwarex/

>From an abstract level, however, citing code requires that it's published
in some form, which is very strongly related to packaging, so I think such
discussions would best revolve around there. Maybe rolling something into
pkg_resources that could pull out a short citation string from some package
metadata (a hypothetical `pkg_resources.get_distribution("numpy").citation`
that could be wrapped by some helper function if desired)? The actual
mechanism to convert metadata into something in the repo (a dunder cite
string in the root module, a separate metadata file, etc.) into the package
metadata isn't as important as rolling said metadata into something part of
the distribution package like the version or long_description fields. Once
the schema of the citation data is defined, you could add it to the
metadata spec (outgrowth of PEP-566)
https://packaging.python.org/specifications/core-metadata/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Non-intrusive debug logging

[Nick Timkovich]

I think part of the reason that logging appears complicated is because
logging actually is complicated. In the myriad different contexts a Python
program runs (daemon, command line tool, interactively), the logging output
should be going to all sorts of different places. Thus was born handlers.
If you want "all the logs", do you really want all the logs from some
library and the library it calls? Thus was born filters.

For better or worse, the "line cost" of a logging call encourages them to
be useful. That said, I'd maybe like a plugin for my editor that could hide
all logging statements for some "mature" projects so I could try to see the
control flow a bit better.

On Thu, Jan 25, 2018 at 3:03 PM, Greg Ewing 
wrote:

> Steve Barnes wrote:
>
>> I would suggest, however, that if this feature is introduced it be
>> controlled via a run-time switch &/or environment variable which defaults
>> to off.
>>
>
> I disagreew with defaulting it to off. That would encourage
> lazy developers to distribute library code full of #l lines,
> so that when you turn it on to debug something of your own,
> you get swamped with someone else's debugging messages.
>
> --
> Greg
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Preemptive multitasking and asyncio

[Nick Timkovich]

If I'm understanding correctly, the interpreter already does this with
threads. About every 15 milliseconds the interpreter will stop a thread and
see if there are any others to work on, see "Grok the GIL," blog:
https://emptysqua.re/blog/grok-the-gil-fast-thread-safe-python/ or the
PyCon talk: https://www.youtube.com/watch?time_continue=150=7SSYhuk5hmc

On Wed, Jan 24, 2018 at 10:46 AM, Thomas Güttler <
guettl...@thomas-guettler.de> wrote:

> I found a question and answer at Stackoverflow[1] which says
> that asyncio/await is like cooperative multitasking.
>
> My whish is to have preemptive multitasking: The interpreter
> does the yielding. The software developer does not need to
> insert async/await keywords into its source code any more.
>
> AFAIK the erlang interpreter does something like this.
>
> I guess it is impossible to implement this, but it was
> somehow important for me to speak out my which.
>
> What do you think?
>
> Regards,
>   Thomas Güttler
>
>
> [1] https://stackoverflow.com/questions/38865050/is-await-in-
> python3-cooperative-multitasking
>
>
> --
> Thomas Guettler http://www.thomas-guettler.de/
> I am looking for feedback: https://github.com/guettli/pro
> gramming-guidelines
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a dict with the attribute access capability

[Nick Timkovich]

On Wed, Nov 29, 2017 at 7:21 AM, Daniel Moisset 
wrote:

> As reference of prior art, there is https://pypi.python.org/pypi/munch in
> PyPI
>

Box is also popular as it permits deeper nesting:
https://pypi.python.org/pypi/python-box/ https://github.com/cdgriffith/Box
Addict is similar: https://pypi.python.org/pypi/addict/2.1.1
https://github.com/mewwts/addict and I've seen AttrDict a few times in code
I maintain: https://pypi.python.org/pypi/attrdict
https://github.com/bcj/AttrDict

With a cursory search, also found:
* DotMap: https://pypi.python.org/pypi/dotmap
https://github.com/drgrib/dotmap
* EasyDict https://pypi.python.org/pypi/easydict
https://github.com/makinacorpus/easydict
* Treedict (older) https://pypi.python.org/pypi/treedict
* Bunch (older) https://pypi.python.org/pypi/bunch

...I spy a theme :P

Haven't dug into them that much, but I'd try to answer most questions with
how they do it. I'm not sure if the recursive 'items as attributes within
items as attributes...' feature is a can of worms or if it's something like
defaultdict(dict), but with more levels. Getting deeply nested items with
dots seems to be the usual use (i.e. complicated JSONs), and I think the
degree of laziness in creating/setting nested items is variable.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using an appropriate tone in emails (was: Adding a thin wrapper class around the functions in stdlib.heapq)

[Nick Timkovich]

On Mon, Nov 27, 2017 at 8:17 PM, Brett Cannon  wrote:

> But calling it "atrocious" and so bad that it needs to be fixed
> "immediately" as if it's a blight upon the stdlib is unnecessarily
> insulting to those that have worked on the module. To convey the feeling
> that you think an OO wrapper would be helpful as the current design doesn't
> work for you, you could just phrase it as I just did to get the same point
> across without insulting anyone. Basically if you wouldn't like your own
> work called "atrocious" by someone you respect, then it's probably best to
> not use that phrasing when talking about a stranger's code either.
>

Sorry for the curt tone, I did lose some sight on the code being designed
by people rather than a faceless organization. My intention wasn't to
disparage the original authors but sprung more out of my frustration and
perception from that thread and those before that the status quo would not
change and that if a contribution was proffered, would simply be dismissed
or ignored. To motivate any change, there must be some argument levied
against the status quo, but hopefully I can articulate it better.

That little corner is something I'm interested in, and not having
contributed to CPython before, I'm unsure how it "really works". The steps
at https://devguide.python.org/stdlibchanges/ suggest trying to elicit
community feedback from the lists as a step, so negative feedback tends to
kill the enthusiasm to actually make the PR. In the absence of code,
concrete arguments are almost impossible as we're discussing the shape of
clouds.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding a thin wrapper class around the functions in stdlib.heapq

[Nick Timkovich]

On Tue, Nov 21, 2017 at 6:45 PM, Steven D'Aprano <st...@pearwood.info>
wrote:

> On Tue, Nov 21, 2017 at 04:56:27PM -0600, Nick Timkovich wrote:
> > On Tue, Nov 21, 2017 at 4:16 PM, Sven R. Kunze <srku...@mail.de> wrote:
> >
> > > Maybe, that suffices: https://pypi.python.org/pypi/xheap
> > >
> > I still think the heapq.heap* functions are atrocious and they should
> > immediately be packaged with *no additional features* into a stdlib
> object
> > for reasons along the line of
> > https://mail.python.org/pipermail/python-ideas/2017-October/047514.html
>
> I think you pasted the wrong URL. That link is about pip, and the
> discoverability of third-party libraries.
>

Probably straining the reference, but akin to "don't teach JSON with
Python, teach XML-RPC because it supports that better in the stdlib", why
not "don't teach heaps, just use lists with pop and insert+sort".


> But generally, Python's APIs are not "pure object oriented" in the Java
> sense, and we don't needlessly create objects just for the sake of
> ensuring everything is an object. Functions are fine too...


Functions are great. I'm a big fan of functions. However,

1. Once there are several that all have the same thing as an argument:
thing_operation1(thing, arg), thing_operation2(thing, arg)...it's about
time to bind them together.
2. And especially for the heap "soft-datatype": once it's heapified,
naively modifying it with other methods will ruin the heap invariant. **The
actual list you pass around can't be treated as a list.**
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding a thin wrapper class around the functions in stdlib.heapq

[Nick Timkovich]

On Tue, Nov 21, 2017 at 4:16 PM, Sven R. Kunze  wrote:

> Maybe, that suffices: https://pypi.python.org/pypi/xheap
>
I still think the heapq.heap* functions are atrocious and they should
immediately be packaged with *no additional features* into a stdlib object
for reasons along the line of
https://mail.python.org/pipermail/python-ideas/2017-October/047514.html

If the improvements in xheap are converging on boringly-stable, maybe bring
them in at a later date.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make bytes __repr__ and __str__ representation different?

[Nick Timkovich]

On Tue, Nov 21, 2017 at 11:22 AM, Chris Barker 
wrote:

> supposedly __repr__ is supposed to give an eval-able version -- which your
> proposal is. But the way you did your example indicates that:
>
> bytes((42, 43, 44, 45, 46))
>
> would be an even better __repr__, if the goal is to make it clear and easy
> that it is a "container of integers from 0 to 255"
>

I wonder if for repr-synonyms, a format specifier to `repr()` could toggle
how the object chooses to display itself would be handy:

x = b'*+-./'
repr(x) # b'*+-./'
repr(x, bytes.REPR_HEX_STRING) # b'\x2a\x2b\x2c\x2d\x2e'
repr(x, bytes.REPR_BYTES) # bytes([42, 43, 44, 45, 46])
repr(x, bytes.REPR_HEX_BYTES) # bytes([0x2A, 0x2B, 0x2C, 0x2D, 0x2E])

Kinda like `format()` but such that all of `eval(repr(x, ))` are
equal.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Nick Timkovich]

I guess for reference:

exec('a\N{MIDDLE DOT} = 0')
exec('\N{BUHID LETTER RA} = 1')
exec('\N{HANGUL LETTER EU} = 2')
exec('\N{TIFINAGH LETTER YO} = 3')
exec('\N{BOPOMOFO LETTER I} = 4')
exec('\N{HANGUL LETTER ARAEA} = 5')

On Sun, Nov 19, 2017 at 1:38 AM, Serhiy Storchaka 
wrote:

> 19.11.17 04:01, Mikhail V пише:
>
>> Python allows underscore character as a separator in variables.
>> This is better than nothing, still it does not make the look much better.
>>
>> **Proposal**: allow additional separator, namely hyphen character.
>>
>
> You already can use "separators" different from the underscore.
>
> my·variable
> myᝍvariable
> myㅡvariable
> myⵧvariable
> myㄧvariable
> myㆍvariable
>
> Be lucky to distinguish one "separator" from other.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Nick Timkovich]

Python does not use U+2010 HYPHEN for the minus operator, it uses the
U+002D (-) HYPHEN-MINUS.

In some monospace fonts, there is a subtle difference between U+002D,
U+2013 EN DASH, and U+2014 EM DASH, but it's usually hard to tell them
*all* apart.

If you want to make a proposal, I'd suggest that you limit it to allowing
the U+2010 HYPHEN to be used for names. U+002D simply cannot be changed
because it would break billions of lines of code.

On Sat, Nov 18, 2017 at 10:44 PM, Mikhail V  wrote:

> On Sun, Nov 19, 2017 at 3:42 AM, Nick Coghlan  wrote:
>
> > For anyone tempted to suggest "What about multiple underscores
> > indicating continuation of the variable name?", that's still a
> > compatibility problem due to the unary minus operator:
> >
> > >>> my--variable
> > 2
> > >>> my---variable
> > 0
>
> That seems to be another showcase of misfotune that Python
> uses hyphen for minus operator. I know it is not language designer's
> fault, because basic ASCII simply did not not include minus character.
> But do you realise that the **current** problem you are adressing is that
> font designers forgot to make the minus character (in monospaced font)
> distinctive from the hyphen character?
>
> Well, what can I say, I just think it should be a reason to make a
> collective complain to font providers, but not that you should silently
> accept this and adopt the language design to someone's sloppy font design.
>
> As an aid for monospace die-hards, to minimise the confusion one could
> publish a style-guide that recommends to disclose the minus operator
> (currently
> hyphen char)  in spaces, like a - b, and probably disallow the new proposed
> hyphen character in the beginning of the identifiers.
> That would still leave potential for confusion because you cant' force
> everyone
> to follow style-guides, but one should struggle to break from this cycle
> anyway.
>
> >
> > Would hyphens in variable names improve readability sometimes?
>
> For reading code, indeed, always and very much. Of course not in case
> I would be forced
> to use monospaced font with a similar minus and hyphen. But
> in that case I am already accepting the level of readability of
> 12th century, so this would not make things much worse, and I
> would simply put spaces around the minus operator and try to highlight
> it with some strong color.
>
>
>
> Mikhail
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Nick Timkovich]

On Sat, Nov 4, 2017 at 10:44 AM, M.-A. Lemburg  wrote:

> Just to clarify: Python 2.0 was called 2.0 because the BeOpen marketing
> department thought it was good idea, not because there were major
> incompatible changes going into that release.
>

Alternative history question: if it was just 1.6, then would Python 3000,
probably called "Python 2000/2.x" in that world, with all the fixes have
happened sooner because you would have "ran out" (yes, v1.10 > v1.9) of
numbers leading up to 2?


> With that approach, the versioning scheme would just be an
> implementation detail, which is good. Whether we call it Python
> 4.2 or Python 42 is really not all that important. Python
> is mature enough to not have to pull marketing tricks anymore.


But of course we'll skip Python 9, just like iPhones and Windowses ;)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make map() better

[Nick Timkovich]

On Wed, Sep 13, 2017 at 10:54 AM, Jason H  wrote:
>
> Thanks for the insights.
> I don't think it would be that breaking:
>
> def remap_map(a1, a2):
>   if hasattr(a1, '__call__'):
> return map(a1, a2)
>   elif hasattr(a2, '__call__'):
> return map(a2,a1)
>   else:
> raise NotCallable # Exception neither is callable
>

I think it's better to be parsimonious and adhere to the "there is one way
to do it" design principle. On the matter of style, map with a lambda is
more pleasing as `(expr-x for x in iterable)` rather than `map(lambda x:
expr-x, iterable)`. If you need to handle multiple iterables, they can be
zip'd.


> I'm rather surprised that there isn't a Iterable class which dict and list
> derive from.
> If that were added to just dict and list, I think it would cover 98% of
> cases, and adding Iterable would be reasonable in the remaining scenarios.


For checking, there's `collections.abc.Iterable` and neighbors that can
look at the interface easily, but I don't think the C-implemented, built-in
types spring from them.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Idea : for smarter assignment?

[Nick Timkovich]

On Fri, Jul 21, 2017 at 12:59 PM, David Mertz  wrote:
>
> But you've left out quite a few binding operations.  I might forget some,
> but here are several:
>

Ned Batchelder had a good presentation at PyCon 2015 about
names/values/assignments/binding: https://youtu.be/_AEJHKGk9ns?t=12m52s His
summary of all assignment operators:

X = ...
for X in ...
class X: pass
def X: pass
def fn(X): # when called, X is bound
import X
from ... import X
except ... as X:
with ... as X:

...I think only includes one other assignment type from what you listed
(function parameters) that ironically is where one could maybe blur =/:, as
doing f(x=3) and f(**{x: 3}) are usually similar (I think some C functions
react poorly?).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] CPython should get...

[Nick Timkovich]

On Sat, Jul 1, 2017 at 1:17 PM, Oleg Broytman  wrote:
>
>I think the sentence "Python should have  implement feature>" should be ;-) forbidden if it is not followed with
> "I'm in the middle of development. Expect the 1st PR in  timeframe>."
>
>Python can only have features that You, the , implemented (or
> paid for) and submitted.
>

Devil's advocate: why prepare a patch and submit it if it is going to be
dismissed out of hand. Trying to gauge support for the idea is a reasonable
first-step.

Devil's devil's advocate: if it has value, it could stand on it's own and
gain it's own group of supporters as a CPython fork.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Binary arithmetic does not always call subclasses first

[Nick Timkovich]

GitHub shows that that note hasn't changed in 10 years:
https://github.com/python/cpython/blame/master/Doc/reference/datamodel.rst#L2210

On Mon, Apr 24, 2017 at 3:15 PM, Terry Reedy  wrote:

> On 4/24/2017 12:14 PM, Stephan Hoyer wrote:
>
> Based on the change in the documentation between 2.x and 3.x, I wonder if
>> this is something that someone intended to clean up as part of Python 3000
>> but never got around to. I would love to hear from anyone familiar with the
>> historical context here.
>>
>
> We should ask the intention of the person who made the change, which is in
> the repository.  If you email me the doc (the last part of the url) and
> location within, I will look it up.
>
> --
> Terry Jan Reedy
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add pathlib.Path.write_json andpathlib.Path.read_json

[Nick Timkovich]

>
> I attended a talk at PYCON UK that talked to the point of using object
> composition
> rather then rich interfaces. I cannot recall the term that was used to
> cover this idea.
>

>

Separating things by concern/abstraction (the storage vs. the
serialization) results in easier-to-learn code, *especially* incrementally,
as you can (for example) plug reading from a file, a socket, a database
into the same JSON, INI, XML... functions.

Learn N ways to read data, M ways to transform the data, and you can do N*M
things with N+M knowledge. If the libraries start tightly coupling
everything, you need to start going through N*M methods, then do it
yourself anyways, because reader X doesn't support new-hotness-format Y
directly.

Perhaps less code could result from making objects "quack" alike, so
instead of you doing the plumbing, the libraries themselves would. I
recently was satisfied by being able to exchange

with open('dump.txt') as f:
for line in f:...

with

import gzip
with gzip.open('dump.gz', 'rt') as f:
for line in f:...

and it just worked through the magic of file-like objects and context
managers.

Nick
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

[Nick Timkovich]

I see the benchmarks, and while I assume the asymptotic complexity is the
same, is there a longer "start-up" time your optimizations need? Do you
have benchmarks where you sort 10, 100...10**6 items to show that beyond
the types you're sorting, you're not amortizing any increased overhead out
to oblivion?

On Sun, Mar 5, 2017 at 9:24 PM, Elliot Gorokhovsky <
elliot.gorokhov...@gmail.com> wrote:

> On Sun, Mar 5, 2017 at 8:20 PM David Mertz  wrote:
>
>>
>> B. I think a very large percentage of lists are heterogeneous.  But most
>> of the time when they are, it's not because there are several different
>> numeric types but rather because a list collects some sort of custom
>> objects.  Maybe those even all share some ancestor, but the point is they
>> are user/library defined classes that won't have a fast comparison like
>> ints or floats do.
>>
>
> As long as the ob_type for all the objects is the same, my patch will sort
> significantly faster, as it replaces PyObject_RichCompareBool with
> ob_type->tp_richcompare. It doesn't matter if your list is builtin-types or
> not, as long as the types are all the same, you get a speedup (though it's
> greater for built-in types).
>
> Also, I actually wrote a crawler to see how many PyPI packages implement a
> custom compare function (like you would have to for user-defined types).
> The answer is less than 6%. Code: https://github.com/embg/
> python-fast-listsort/tree/master/crawler
>
>
>> B (part 2): I haven't actually read his patch, but I'm assuming that
>> Elliot's approach can start scanning the list, and as soon as it find a
>> complex/custom object at index 0 ignore the rest of the scan.  So for that
>> case, there is very little harm in his linear scan (over one item).
>>
>
> Yup, the pre-sort check breaks out of the loop as soon as it sees
> heterogeneity. So unless your float is at the end of the whole list of ints
> (as in my worst-case benchmark), the cost is basically 0.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] suggestion about the sort() function of the list instance

[Nick Timkovich]

>From my experience teaching Python to non-programmers, it's a huge
hurdle/nightmare to teach functions/methods that modify objects in-place
vs. return a value that must be reassigned. Behold Pandas's DataFrame's
sort method, which has an optional `in_place` argument that defaults to
*False*, which despite being a method that looks like mylist.sort(), works
differently from the method on lists, but more like the sorted *function*,
arrrgh! That was a fun session...

I think for consistency, having object methods that can act in-place
(semantics like mylist.pop, mylist.append are nice as Stéfane suggests)
only act in-place, and functions return a new object for reassignment would
help new users.

Maybe I'm teaching it poorly, suggestions welcome.

Nick

On Wed, Mar 1, 2017 at 11:26 AM, Stéfane Fermigier  wrote:

> Definitively not, just like M. Fowler: "Meyer likes to use command-query
> separation absolutely, but there are exceptions. Popping a stack is a good
> example of a query that modifies state. Meyer correctly says that you can
> avoid having this method, but it is a useful idiom. So I prefer to follow
> this principle when I can, but I'm prepared to break it to get my pop."
>
> What I wanted to point out is that the paragraph quoted by Stephan ("In
> general in Python (and in all cases in the standard library) a method that
> mutates an object will return None to help avoid getting the two types of
> operations confused. So if you mistakenly write y.sort() thinking it will
> give you a sorted copy of y, you’ll instead end up with None, which will
> likely cause your program to generate an easily diagnosed error.") doesn't
> seem to be true in this case.
>
>   S.
>
> On Wed, Mar 1, 2017 at 6:23 PM, Cory Benfield  wrote:
>
>>
>> On 1 Mar 2017, at 10:26, Stéfane Fermigier  wrote:
>>
>> Cf. https://martinfowler.com/bliki/CommandQuerySeparation.html
>>
>> But:
>>
>> >>> l = [1,2,3]
>> >>> l.pop()
>> 3
>> >>> l
>> [1, 2]
>>
>> => Not so true.
>>
>>   S.
>>
>>
>> This is naturally a different circumstance: pop must return the element
>> it popped, otherwise it would just be del. Surely you aren’t suggesting
>> that pop should return self?
>>
>> Cory
>>
>
>
>
> --
> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier
> - http://linkedin.com/in/sfermigier
> Founder & CEO, Abilian - Enterprise Social Software -
> http://www.abilian.com/
> Chairman, Free Group / Systematic Cluster -
> http://www.gt-logiciel-libre.org/
> Co-Chairman, National Council for Free & Open Source Software (CNLL) -
> http://cnll.fr/
> Founder & Organiser, PyData Paris - http://pydata.fr/
> ---
> “You never change things by fighting the existing reality. To change
> something, build a new model that makes the existing model obsolete.” —
> R. Buckminster Fuller
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Contraction for "for x in range()"

[Nick Timkovich]

Make some shim object that you can index into to get that functionality,
could even call it Z (for the set of all integers). Short, and requires no
new syntax.

class IndexableRange:
def __getitem__(self, item):
if isinstance(item, slice):
start = item.start if item.start is not None else 0
step = item.step if item.step is not None else 1
if item.stop is None:
return itertools.count(start, step)
else:
return range(start, item.stop, step)
else:
return item

Z = IndexableRange()

for y in Z[0:10:2]:
print(y)

On Tue, Feb 14, 2017 at 5:22 PM, Mikhail V  wrote:

>
> On 14 February 2017 at 22:41, MRAB  wrote:
>
>> On 2017-02-14 21:09, Zachary Ware wrote:
>>
>>> On Tue, Feb 14, 2017 at 3:06 PM, Mikhail V  wrote:
>>>
 I have a small syntax idea.
 In short, contraction of

 for x in range(a,b,c) :

 to

 for x in a,b,c :

 I really think there is something cute in it.
 So like a shortcut for range() which works only in for-in statement.
 So from syntactical POV, do you find it nice syntax?
 Visually it seems to me less bulky than range().

 Example:

 for x in 0,5 :
 print (x)
 for y in 0,10,2 :
 print (y)
 for z in 0, y+8 :
 print (z)

>>>
>>> This is already valid and useful syntax, and thus a non-starter.
>>>
>>>
>>>
>>> The closest you could get without breaking existing code is [a:b:c]:
>>
>> for x in [0:5]:
>> print(x)
>> for y in [0:10:2]:
>> print(y)
>> for z in [0:y+8]:
>> print(z)
>>
>> What's more, that could be valid outside the 'for' loop too.
>>
>>
> This looks really good, at first glance I would even expect this will
> work,
> comes so common from lists and numpy index ranges.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Contraction for "for x in range()"

[Nick Timkovich]

for i in range(...) is *sometimes* indicative of code smell, especially
when then doing x[i], though it has its uses. I've never had a need to
shorten a for...range line though.

Other than it being "cute", do you have an example where it's definitively
better?

On Tue, Feb 14, 2017 at 4:03 PM, Todd  wrote:

> On Tue, Feb 14, 2017 at 4:41 PM, MRAB  wrote:
>
>> On 2017-02-14 21:09, Zachary Ware wrote:
>>
>>> On Tue, Feb 14, 2017 at 3:06 PM, Mikhail V  wrote:
>>>
 I have a small syntax idea.
 In short, contraction of

 for x in range(a,b,c) :

 to

 for x in a,b,c :

 I really think there is something cute in it.
 So like a shortcut for range() which works only in for-in statement.
 So from syntactical POV, do you find it nice syntax?
 Visually it seems to me less bulky than range().

 Example:

 for x in 0,5 :
 print (x)
 for y in 0,10,2 :
 print (y)
 for z in 0, y+8 :
 print (z)

>>>
>>> This is already valid and useful syntax, and thus a non-starter.
>>>
>>>>>> for x in 0,5 :
>>>... print (x)
>>>... for y in 0,10,2 :
>>>... print (y)
>>>... for z in 0, y+8 :
>>>... print (z)
>>>...
>>>0
>>>0
>>>0
>>>8
>>>10
>>>0
>>>18
>>>2
>>>0
>>>10
>>>5
>>>0
>>>0
>>>8
>>>10
>>>0
>>>18
>>>2
>>>0
>>>10
>>>
>>> The closest you could get without breaking existing code is [a:b:c]:
>>
>> for x in [0:5]:
>> print(x)
>> for y in [0:10:2]:
>> print(y)
>> for z in [0:y+8]:
>> print(z)
>>
>> What's more, that could be valid outside the 'for' loop too.
>>
>>
> Guido has already rejected this syntax and several others.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] globals should accept parenteses for extending beyond 1 line

[Nick Timkovich]

Related and probably more common is the need for the line-continuation
operator for long/multiple context managers with "with". I assume that's
come up before, but was it also just a low priority rather than any
technical reason?

On Mon, Jan 23, 2017 at 1:53 PM, Brett Cannon  wrote:

> Actually multi-line import doesn't work:
>
> File ".\Untitled.py", line 1
> import (tokenize,
>^
> SyntaxError: invalid syntax
>
> I think you're getting this mixed up with parentheses being allowed in
> `from ... import (...)` syntax. So unless there is another single-word
> keyword that allows multi-line arguments using parentheses I don't think
> there's an inconsistency here.
>
> Plus, as Guido pointed out, the current syntax isn't preventing you from
> doing something you can already do. So if you want to add parentheses
> support to global, nonlocal, and import, you can propose a patch, but it's
> not a priority to solve without someone providing a solution since it
> doesn't open up anything new for something people don't use on a regular
> basis.
>
>
> On Mon, 23 Jan 2017 at 11:39 João Matos  wrote:
>
>> Hello,
>>
>> You are correct, my mistake. I should have written global and not globals.
>>
>> The purpose of using parentheses on the import statement is not (in my
>> view) for operational efficiency but for appearance/cleaness.
>> The same applies to using it to global.
>>
>> One does not need to have 10 global vars. It may have to do with var
>> name length and the 79 max line length.
>>
>> This is an example from my one of my programs:
>> global existing_graph, expected_duration_in_sec, file_size, \
>>  file_mtime, no_change_counter
>>
>> Anyway, the use of global being rare is of no concern. The point of my
>> suggestion is standardization.
>> My opinion is that a standard language is easier to learn (and teach)
>> than one that has different syntax for the same issue, depending on the
>> statement.
>>
>> In short, if the recommended multi-line use for import is
>>
>> import (a, b,
>>  c)
>>
>> instead of
>>
>> import a, b, \
>>  c
>>
>> Then the same should apply to global.
>>
>>
>> Best regards,
>>
>> JM
>>
>>
>>
>>
>> On 23-01-2017 19:25, Terry Reedy wrote:
>> > On 1/23/2017 1:43 PM, João Matos wrote:
>> >> Hello,
>> >>
>> >> I would like to suggest that globals should follow the existing rule
>> >> (followed by the import statement, the if statement and in other
>> places)
>> >> for extending beyond 1 line using parentheses.
>> >> Like this:
>> >> globals (var_1, var_2,
>> >> var_3)
>> >>
>> >> instead of what must be done now, which is:
>> >> globals var_1, var_2 \
>> >> var_3
>> >
>> > The declaration keyword is 'global'; 'globals' is the built-in
>> > function.  In any case
>> >
>> > global var_1, var_2
>> > global var_3
>> >
>> > works fine.  There is no connection between the names and, unlike with
>> > import, no operational efficiency is gained by mashing the statements
>> > together.
>> >
>> > This issue should be rare.  The global statement is only needed when
>> > one is rebinding global names within a function*.  If a function
>> > rebinds 10 different global names, the design should probably be
>> > re-examined.
>> >
>> > * 'global' at class scope seems useless.
>> >
>> > a = 0
>> > class C:
>> > a = 1
>> >
>> > has the same effect as
>> > a = 0
>> > a = 1
>> > class C: pass
>> >
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Ideas for improving the struct module

[Nick Timkovich]

Construct has radical API changes and should remain apart. It feels to me
like a straw-man to introduce a large library to the discussion as
justification for it being too-specialized.

This proposal to me seems much more modest: add another format character
(or two) to the existing set of a dozen or so that will be packed/unpacked
just like the others. It also has demonstrable use in various
formats/protocols.

On Thu, Jan 19, 2017 at 12:50 PM, Nathaniel Smith <n...@pobox.com> wrote:

> I haven't had a chance to use it myself yet, but I've heard good things
> about
>
> https://construct.readthedocs.io/en/latest/
>
> It's certainly far more comprehensive than struct for this and other
> problems.
>
> As usual, there's some tension between adding stuff to the stdlib versus
> using more specialized third-party packages. The existence of packages like
> construct doesn't automatically mean that we should stop improving the
> stdlib, but OTOH not every useful thing can or should be in the stdlib.
>
> Personally, I find myself parsing uleb128-prefixed strings more often than
> u4-prefixed strings.
>
> On Jan 19, 2017 10:42 AM, "Nick Timkovich" <prometheus...@gmail.com>
> wrote:
>
>> ctypes.Structure is *literally* the interface to the C struct that as
>> Chris mentions has fixed offsets for all members. I don't think that should
>> (can?) be altered.
>>
>> In file formats (beyond net protocols) the string size + variable length
>> string motif comes up often and I am frequently re-implementing the
>> two-line read-an-int + read-{}.format-bytes.
>>
>> On Thu, Jan 19, 2017 at 12:17 PM, Joao S. O. Bueno <jsbu...@python.org.br
>> > wrote:
>>
>>> I am for upgrading struct to these, if possible.
>>>
>>> But besides my +1,  I am writting in to remember folks thatthere is
>>> another
>>> "struct" model in the stdlib:
>>>
>>> ctypes.Structure  -
>>>
>>> For reading a lot of records with the same structure it is much more
>>> handy than
>>> struct, since it gives one a suitable Python object on instantiation.
>>>
>>> However, it also can't handle variable lenght fields automatically.
>>>
>>> But maybe, the improvement could be made on that side, or another package
>>> altogether taht works more like it than current "struct".
>>>
>>>
>>>
>>> On 19 January 2017 at 16:08, Elizabeth Myers <elizab...@interlinked.me>
>>> wrote:
>>> > On 19/01/17 06:47, Elizabeth Myers wrote:
>>> >> On 19/01/17 05:58, Rhodri James wrote:
>>> >>> On 19/01/17 08:31, Mark Dickinson wrote:
>>> >>>> On Thu, Jan 19, 2017 at 1:27 AM, Steven D'Aprano <
>>> st...@pearwood.info>
>>> >>>> wrote:
>>> >>>>> [...] struct already supports
>>> >>>>> variable-width formats.
>>> >>>>
>>> >>>> Unfortunately, that's not really true: the Pascal strings it
>>> supports
>>> >>>> are in some sense variable length, but are stored in a fixed-width
>>> >>>> field. The internals of the struct module rely on each field
>>> starting
>>> >>>> at a fixed offset, computable directly from the format string. I
>>> don't
>>> >>>> think variable-length fields would be a good fit for the current
>>> >>>> design of the struct module.
>>> >>>>
>>> >>>> For the OPs use-case, I'd suggest a library that sits on top of the
>>> >>>> struct module, rather than an expansion to the struct module itself.
>>> >>>
>>> >>> Unfortunately as the OP explained, this makes the struct module a
>>> poor
>>> >>> fit for protocol decoding, even as a base layer for something.  It's
>>> one
>>> >>> of the things I use python for quite frequently, and I always end up
>>> >>> rolling my own and discarding struct entirely.
>>> >>>
>>> >>
>>> >> Yes, for variable-length fields the struct module is worse than
>>> useless:
>>> >> it actually reduces clarity a little. Consider:
>>> >>
>>> >>>>> test_bytes = b'\x00\x00\x00\x0chello world!'
>>> >>
>>> >> With this, you can do:
>>> >>
>>> >>>>> length = int.from_bytes(test_bytes[:4], 'big')
>>> >>>>> string = test_b

Re: [Python-ideas] Ideas for improving the struct module

[Nick Timkovich]

ctypes.Structure is *literally* the interface to the C struct that as Chris
mentions has fixed offsets for all members. I don't think that should
(can?) be altered.

In file formats (beyond net protocols) the string size + variable length
string motif comes up often and I am frequently re-implementing the
two-line read-an-int + read-{}.format-bytes.

On Thu, Jan 19, 2017 at 12:17 PM, Joao S. O. Bueno 
wrote:

> I am for upgrading struct to these, if possible.
>
> But besides my +1,  I am writting in to remember folks thatthere is another
> "struct" model in the stdlib:
>
> ctypes.Structure  -
>
> For reading a lot of records with the same structure it is much more handy
> than
> struct, since it gives one a suitable Python object on instantiation.
>
> However, it also can't handle variable lenght fields automatically.
>
> But maybe, the improvement could be made on that side, or another package
> altogether taht works more like it than current "struct".
>
>
>
> On 19 January 2017 at 16:08, Elizabeth Myers 
> wrote:
> > On 19/01/17 06:47, Elizabeth Myers wrote:
> >> On 19/01/17 05:58, Rhodri James wrote:
> >>> On 19/01/17 08:31, Mark Dickinson wrote:
>  On Thu, Jan 19, 2017 at 1:27 AM, Steven D'Aprano  >
>  wrote:
> > [...] struct already supports
> > variable-width formats.
> 
>  Unfortunately, that's not really true: the Pascal strings it supports
>  are in some sense variable length, but are stored in a fixed-width
>  field. The internals of the struct module rely on each field starting
>  at a fixed offset, computable directly from the format string. I don't
>  think variable-length fields would be a good fit for the current
>  design of the struct module.
> 
>  For the OPs use-case, I'd suggest a library that sits on top of the
>  struct module, rather than an expansion to the struct module itself.
> >>>
> >>> Unfortunately as the OP explained, this makes the struct module a poor
> >>> fit for protocol decoding, even as a base layer for something.  It's
> one
> >>> of the things I use python for quite frequently, and I always end up
> >>> rolling my own and discarding struct entirely.
> >>>
> >>
> >> Yes, for variable-length fields the struct module is worse than useless:
> >> it actually reduces clarity a little. Consider:
> >>
> > test_bytes = b'\x00\x00\x00\x0chello world!'
> >>
> >> With this, you can do:
> >>
> > length = int.from_bytes(test_bytes[:4], 'big')
> > string = test_bytes[4:length]
> >>
> >> or you can do:
> >>
> > length = struct.unpack_from('!I', test_bytes)[0]
> > string = struct.unpack_from('{}s'.format(length), test_bytes, 4)[0]
> >>
> >> Which looks more readable without consulting the docs? ;)
> >>
> >> Building anything on top of the struct library like this would lead to
> >> worse-looking code for minimal gains in efficiency. To quote Jamie
> >> Zawinksi, it is like building a bookshelf out of mashed potatoes as it
> >> stands.
> >>
> >> If we had an extension similar to netstruct:
> >>
> > length, string = struct.unpack('!I$', test_bytes)
> >>
> >> MUCH improved readability, and also less verbose. :)
> >
> > I also didn't mention that when you are unpacking iteratively (e.g., you
> > have multiple strings), the code becomes a bit more hairy:
> >
>  test_bytes = b'\x00\x05hello\x00\x07goodbye\x00\x04test'
>  offset = 0
>  while offset < len(test_bytes):
> > ... length = struct.unpack_from('!H', test_bytes, offset)[0]
> > ... offset += 2
> > ... string = struct.unpack_from('{}s'.format(length), test_bytes,
> > offset)[0]
> > ... offset += length
> >
> > It actually gets a lot worse when you have to unpack a set of strings in
> > a context-sensitive manner. You have to be sure to update the offset
> > constantly so you can always unpack strings appropriately. Yuck!
> >
> > It's worth mentioning that a few years ago, a coworker and I found
> > ourselves needing variable length strings in the context of a binary
> > protocol (DHCP), and wound up abandoning the struct module entirely
> > because it was unsuitable. My co-worker said the same thing I did: "it's
> > like building a bookshelf out of mashed potatoes."
> >
> > I do understand it might require a possible major rewrite or major
> > changes the struct module, but in the long run, I think it's worth it
> > (especially because the struct module is not all that big in scope). As
> > it stands, the struct module simply is not suited for protocols where
> > you have variable-length strings, and in my experience, that is the vast
> > majority of modern binary protocols on the Internet.
> >
> > --
> > Elizabeth
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> 

Re: [Python-ideas] AtributeError inside __get__

[Nick Timkovich]

Are you saying that hasattr returning False was hiding a bug or is a bug?
The former could be annoying to track down, though hasattr(X, 'y') == True.
For the latter, having hasattr return False if an AttributeError is raised
would allow the property decorator to retain identical functionality if it
is used to replace a (sometimes) existing attribute.

On Sun, Dec 25, 2016 at 1:24 PM, Zahari Dim  wrote:

> Hi,
>
> The other day I came across a particularly ugly bug. A simplified case
> goes like:
>
> class X:
> @property
> def y(self):
> return self.nonexisting
>
> hasattr(X(),'y')
>
> This returns False because hasattr calls the property which in turn
> raises an AttributeError which is used to determine that the property
> doesn't exist, even if it does. This is arguably unexpected and
> surprising and can be very difficult to understand if it happens
> within a large codebase. Given the precedent with generator_stop,
> which solves a similar problem for StopIteration, I was wondering if
> it would be possible to have the __get__ method convert the
> AttributeErrors raised inside it to RuntimeErrors.
>
> The situation with this is a little more complicated because there
> could be a (possibly strange) where one might want to raise an
> AttributeError inside __get__. But maybe the specification can be
> changed so either `raise ForceAttributeError()` or `return
> NotImplemented` achieves the same effect.
>
>
> Merry Christmas!
> Zahari.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Input characters in strings by decimals (Was: Proposal for default character representation)

[Nick Timkovich]

Out of curiosity, why do you prefer decimal values to refer to Unicode code
points? Most references, http://unicode.org/charts/PDF/U0400.pdf (official)
or https://en.wikibooks.org/wiki/Unicode/Character_reference/-0FFF ,
prefer to refer to them by hexadecimal as the planes and ranges are broken
up by hex values.

On Wed, Dec 7, 2016 at 5:52 PM, Mikhail V  wrote:

> In past discussion about inputing and printing characters,
> I was proposing decimal notation instead of hex.
> Since the discussion was lost in off-topic talks, I'll try to
> summarise my idea better.
>
> I use ASCII only for code input (there are good reasons for that).
> Here I'll use Python 3.6, and Windows 7, so I can use print() with unicode
> directly and it works now in system console.
>
> Suppose I only start programming and want to do some character
> manipulation.
> The vey first thing I would probably start with is a simple output for
> latin and cyrillic capital letters:
>
> caps_lat = ""
> for o in range(65, 91):
> caps_lat =  caps_lat + chr(o)
> print (caps_lat)
>
> caps_cyr = ""
> for o in range(1040, 1072):
> caps_cyr =  caps_cyr + chr(o)
> print (caps_cyr)
>
>
> Which prints:
> ABCDEFGHIJKLMNOPQRSTUVWXYZ
> АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ
>
>
> Say, I want now to input something direct in code:
>
> s = "first cyrillic letters: " + chr(1040) + chr(1041) + chr(1042)
>
> Which works fine and has clean look. However it is not very convinient
> because of much typing and also, if I generate such strings,
> adds a bit more complexity. But in general it is fine, and I use this
> method currently.
>
> =
> Proposal: I would want to have a possibility to input it *by decimals*:
>
> s = "first cyrillic letters: \{1040}\{1041}\{1042}"
> or:
> s = "first cyrillic letters: \(1040)\(1041)\(1042)"
>
> =
>
> This is more compact and seems not very contradictive with
> current Python escape characters in string literals.
> So backslash is a start of some escaping in most cases.
>
> For me most important is that in such way I would avoid
> any presence of hex numbers in strings, which I find very good
> for readability and for me it is very convinient since I use decimals
> for processing everywhere (and encourage everyone to do so).
>
> So this is my proposal, any comments on this are appreciated.
>
>
> PS:
>
> Currently Python 3 supports these in addition to \x:
> (from https://docs.python.org/3/howto/unicode.html)
> """
> If you can’t enter a particular character in your editor or want to keep
> the source code ASCII-only for some reason, you can also use escape
> sequences in string literals.
>
> >>> "\N{GREEK CAPITAL LETTER DELTA}"  # Using the character name
> >>> "\u0394"  # Using a 16-bit hex value
> >>> "\U0394"  # Using a 32-bit hex value
>
> """
> So I have many possibilities and all of them strangely contradicts with
> my image of intuitive and readable. Well, using charater name is readable,
> but seriously not much of a practical solution for input, but could be
> very useful
> for printing description of a character.
>
>
> Mikhail
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow random.choice, random.sample to work on iterators

[Nick Timkovich]

Is the goal to allow them to consume a finite generator of *unknown* length
(requires reservoir sampling
https://en.wikipedia.org/wiki/Reservoir_sampling  with N random calls,
which seemed to be the rub before?) or just consume a generator with known
length that's not indexable (a rare beast?). Consuming iterables if they
have a length like below wouldn't be so bad, but might be too niche.

class X:
def __init__(self, ele): self.ele = ele
def __len__(self): return len(self.ele)
def __iter__(self): return iter(self.ele)

x = X([1, 2, 3, 4, 5])
random.choice(x) # TypeError: 'X' object does not support indexing

Would allowing an optional 'len' argument alongside the iterator to
sample/choice be too narrow to be useful?

On Wed, Nov 30, 2016 at 2:21 PM, Bernardo Sulzbach <
mafagafogiga...@gmail.com> wrote:

> On 2016-11-30 17:57, Chris Kaynor wrote:
>
>> On Wed, Nov 30, 2016 at 11:52 AM, Chris Kaynor 
>> wrote:
>>
>>> There are also issues with how it should behave on iterables that
>>> cannot be re-iterated (eg, random.choice will consume the iterator,
>>> and could only be called once safely).
>>>
>>
>> I meant to include a sample in my previous e-mail:
>>
>> Consider that this code will not produce the "correct" results (for a
>> reasonable definition of correct):
>>
>> a = (i for i in range(100)) # Pretend this does something more
>> interesting, and isn't a trivial generator - maybe a file object
>> reading by line.
>> randomEntries = [random.choice(a) for i in range(10)]
>>
>
> In such a case you should explicitly use a sample.
>
> I see your example as the caller's fault, which ignored the fact that the
> iterator would change after calls to choice.
>
> Hold the first 10 (in this case). For every subsequent element, randomly
> choose to replace one of the "held" ones by it (with a diminishing
> probability).
>
> Assume this does not offer the same performance as loading everything into
> memory. But it isn't meant to do so, as if you need / can / want, you could
> just shove it all into a list and use what we currently have.
>
> --
> Bernardo Sulzbach
> http://www.mafagafogigante.org/
> mafagafogiga...@gmail.com
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] (no subject)

[Nick Timkovich]

I would consider the speed of the "ultimate error handler" (i.e. whatever
prints the traceback and kills the program) in the interpreter to be moot,
so long as it takes a small fraction of a second. Optimizing Python's speed
it crashes super-fast due to an *unhandled* NameError in your program seems
folly.

Regarding more informative messages for (e.g.) IndexError, would those just
apply to built-in types as they're the most universal, or should some
additional introspection be done for similar ducks?

If it's useful/there's interest, I could try to do some analysis of Python
questions on SO and see what the most common errors are. I'd guess things
like "'NoneType' object has no attribute ..." would probably be up there,
but that's a whole can of worms as to why someone's trying to call a method
on, index, etc. something they accidentally whacked (a = a.sort()).

On Tue, Nov 29, 2016 at 11:42 AM, Chris Barker 
wrote:

> On Tue, Nov 29, 2016 at 5:48 AM, Nick Coghlan  wrote:
>
>> > SyntaxErrors in an inner loop? That seems unlikely to me.
>>
>
> Syntax Errors are a special case, as by definition the code isn't being
> run yet (yes, there could be an eval in there...)
>
> So we could at least make those more informative without worrying about
> performance.
>
> Also -- would it be possible to tack on the more informative message at a
> higher level? Once the Exception bubbles up to the REPL, is there enough
> information available to make a more informative message?
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Decorator to avoid a mistake

[Nick Timkovich]

You can do it at run-time, if you so desire, without a measurable
performance hit with a metaclass. Here's a hacky demo:
https://gist.github.com/nicktimko/5f08d6adfa1dbe1319c3bfc715ec0aa4#file-override_guard-ipynb

(Pedants: Any performance hit will be constant-time and probably less than
a stray import that you don't need. If you're worried about optimizing
constant-time things, I think you have larger problems.)

On Thu, Nov 24, 2016 at 5:48 PM, Victor Stinner 
wrote:

> Similar or related issue recently open and quickly closed:
> http://bugs.python.org/issue28776
> "Duplicate method names should be an error"
>
> In short, such job should be done by linters. I'm quite sure that many
> of them already implement such check.
>
> Victor
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Decorator to avoid a mistake

[Nick Timkovich]

I think you could implement this yourself with metaclasses and it wouldn't
have much (if any) performance hit per-call or per-instantiation (just a
bit slower when creating the class definition).

It's a bit heavy-handed-hand-holding–if you ask me–but if you want to do
it, the language gives you the ability.

On Tue, Nov 22, 2016 at 3:24 PM, Adrián Orive Oneca <
adrian.orive.on...@gmail.com> wrote:

> This is just a convenience utility that would impact performance. This
> kind of enhancements, in my opinion, should be taken care by the IDEs, not
> by the interpreters.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Method signature syntactic sugar (especially for dunder methods)

[Nick Timkovich]

Also you can support future changes to the syntax (e.g. __matmul__ and
friends from 3.5, __aiter__ from 3.5.2) with a single codebase rather than
having to push that grammar back to previous versions (impossible?) or have
the grammar for magic methods be extraordinarily general (messy?)

On Tue, Nov 8, 2016 at 3:56 PM, Greg Ewing 
wrote:

> Another advantage of dunder method names is that you
> can google them. Someone coming across a method called
> "__foo__" can easily find documentation about it, but
> it's not so easy to do that for special syntax.
>
> --
> Greg
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] SI scale factors alone, without units or dimensional analysis

[Nick Timkovich]

Ah, always mess up micro = 6/9 until I think about it for half a second.
Maybe a "n" suffix could have saved me there ;) For "long" numbers there's
the new _ so you can say 0.000_000_1 if you so preferred for 0.1 micro (I
generally see _ as more useful for high-precison numbers with more non-zero
digits, e.g. 1_234_456_789). Would that be 0.1µ, 0.1u in a new system.

Veering a bit away from the 'suffixing SI prefixes for literals': Literal
unary suffix operators might be slightly nicer than multiplication if they
were #1 in operator precedence, then you could omit some parentheses. Right
now if I want to use a unit:

$ pip install quantities
import quantities as pq
F = 1 * pq.N
d = 1 * pq.m
F * d # => array(1.0) * m*N

but with literal operators & functions could be something like

F = 1 pq.N
d = 1 pq.m


On Sat, Oct 29, 2016 at 1:18 PM, Todd <toddr...@gmail.com> wrote:

> On Sat, Oct 29, 2016 at 12:43 PM, Nick Timkovich <prometheus...@gmail.com>
> wrote:
>
>> From that page:
>>
>>> User-defined literals are basically normal function calls with a fancy
>>> syntax. [...] While user defined literals look very neat, they are not much
>>> more than syntactic sugar. There is not much difference between defining
>>> and calling a literal operator with "foo"_bar and doing the same with an
>>> ordinary function as bar("foo"). In theory, we could write literal
>>> operators that have side effects and do anything we want, like a normal
>>> function.
>>
>>
>> Obviously the arbitrary-function-part of that will never happen in Python
>> (yes?)
>>
>>
>>
> Why not?  It seems like that would solve a lot of use-cases.  People get
> bringing up various new uses for prefix or suffix syntax that they want
> built directly into the language.  Providing a generic way to implement
> third-party prefixes or suffixes would save having to put all of these
> directly into the language.  And it opens up a lot of other potential
> use-cases as well.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] SI scale factors alone, without units or dimensional analysis

[Nick Timkovich]

>From that page:

> User-defined literals are basically normal function calls with a fancy
> syntax. [...] While user defined literals look very neat, they are not much
> more than syntactic sugar. There is not much difference between defining
> and calling a literal operator with "foo"_bar and doing the same with an
> ordinary function as bar("foo"). In theory, we could write literal
> operators that have side effects and do anything we want, like a normal
> function.


Obviously the arbitrary-function-part of that will never happen in Python
(yes?)

Also, for discussion, remember to make the distinction between 'units'
(amps, meters, seconds) and 'prefixes' (micro, milli, kilo, mega). Right
away from comments, it seems 1_m could look like 1 meter to some, or 0.001
to others. Typically when I need to enter very small/large literals, I'll
use "engineering" SI notation (powers divisible by 3 that correspond to the
prefixes): 0.1e-9 = 0.1 micro.

On Sat, Oct 29, 2016 at 12:20 AM, Ryan Birmingham 
wrote:

> I'd certainly be interested in hearing about how this has worked with C++,
> but this would certainly make scientific code less easy to misuse due to
> unclear units.
>
> -Ryan Birmingham
>
> On 28 October 2016 at 16:45, Sven R. Kunze  wrote:
>
>> On 28.10.2016 22:06, MRAB wrote:
>>
>>> On 2016-08-26 13:47, Steven D'Aprano wrote:
>>>
 Ken has made what I consider a very reasonable suggestion, to introduce
 SI prefixes to Python syntax for numbers. For example, typing 1K will be
 equivalent to 1000.

 Just for the record, this is what you can now do in C++:
>>>
>>> User-Defined Literals
>>> http://arne-mertz.de/2016/10/modern-c-features-user-defined-literals/
>>>
>>
>> Nice to hear. :)
>>
>> They now have 5 years of experience with that. Are there any surveys,
>> experience reports, etc.?
>>
>>
>> Cheers,
>> Sven
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Heap data type, the revival

[Nick Timkovich]

Functions are great; I'm a big fan of functions. That said, the group
of heapq.heap* functions are literally OOP without using that "class"
word. As far as flexibility, what is the use of the those functions on
non-heap structures?

On Sat, Oct 15, 2016 at 4:21 PM, Sven R. Kunze <srku...@mail.de> wrote:
> On 15.10.2016 23:19, Nick Timkovich wrote:
>>
>> Features and speed are good, but I'm interested in getting something
>> with the basic features into the Standard Library so it's just there.
>> Not having done that before and bit clueless, I'm wanting to learn
>> that slightly less-technical procedure. What are the steps to make
>> that happen?
>
>
> As I said, it has been discussed and the consensus so far was: "not
> everything needs to be a class if it does not provide substantial benefit" +
> "functions are more flexible" + "if it's slower that the original it won't
> happen".
>
> Cheers,
> Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/