[Python-ideas] Proposing additions to the standard library

[Jonathan Crall]

I'm interested in proposing several additions to the Python standard
library, and I would like more information on the procedure for doing so.
Are all additions done via a PEP? If not what is the procedure. If so, I've
read that the first step was to email this board and get feedback.

I have a library called `ubelt` that contains several tools that I think
might be worthy of adding to the standard library.

Here's my bullet point pitch:

   - Python is batteries included. Ubelt contains extra batteries.
   function are extra batteries.
   - Most function in ubelt are fast. All 222 tests takes 7.33 seconds.
   - Ubelt has 100% test coverage (sans `# nocover` locations).
   - I'm only championing a subset of the functions in ubelt. There are
   certainly functions in there that do not belong in the standard library.
   - I have a Jupyter notebook that give a demo of some select functions
   (not necessarily the same as the ones proposed here):
   
https://github.com/Erotemic/ubelt/blob/master/docs/notebooks/Ubelt%20Demo.ipynb
   - I do have documentation (mostly in docstrings) and in the docs folder,
   but I've been having trouble auto-updating read-the-docs. Here is the link
   anyway: https://ubelt.readthedocs.io/en/latest/

Here is a tentative list of interesting functions. Hopefully the names are
descriptive (if not, see docstrings: https://github.com/Erotemic/ubelt)

ub.cmd

ub.compressuser

ub.group_items

ub.dict_hist

ub.find_duplicates

ub.AutoDict

ub.import_module_from_path

ub.import_module_from_name

ub.modname_to_modpath,

ub.modpath_to_modname

ub.ProgIter

ub.ensuredir

ub.expandpath


almost everything in util_list:

allsame, argmax, argmin, argsort, argunique,

chunks, flatten, iter_window, take, unique


These functions might be worth modifying into dictionary methods:

ub.dict_subset

ub.dict_take

ub.map_vals

ub.map_keys

ub.Timerit

ub.Timer



Because I built the library, I tend to like all the functions. Its
difficult to decide if they are stdlib worthy, so there might be some false
positives / negatives.

I'm on the fence about:
CacheStamp, Cacher, NoParam, argflag, argval, dzip, delete, hash_data,
hash_file, memoize, memoize_method, NiceRepr, augpath, userhome,
ensure_app_cache_dir, ensure_app_resource_dir, find_exe, find_path,
get_app_cache_dir, get_app_resource_dir, platform_cache_dir,
 platform_resource_dir, CaptureStdout, codeblock, ensure_unicode, hzcat,
 indent, OrderedSet


Its my hope that some of these are actually useful. Let me know any of the
following: what you think, if there are any questions, if something else
needs to be done, or what the next steps are.

-- 
-Jon
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposing additions to the standard library

[Jonathan Crall]

@Steve, this is just the sort of feedback I was looking for. Small and
conservative additions make sense. I definitely think that some functions
do fit into existing stdlib modules. For instance, AutoDict might go in
collections.

Sorry, some of these aren't descriptive enough, and if you're trying to
> make a pitch for these features.

...

My advice is to collate the functions you want to add into groups of
> related functionality.


Makes sense. I figured that my original list had too may entries to do that
for, or else the email would explode. Separating each small group into its
own thread will allow me to describe the specific function without writing
a novel.

Sometimes there's a good, useful function than doesn't get added because
> there's no reasonable place to put it. For example, a "flatten" function
> has been talked about since Python 1.x days, and we still don't have a
> standard solution for it, because (1) it isn't clear *precisely* what it
> should do, and (2) it isn't clear where it should go.


The flatten example is good to know about. Is there a link to this
discussion or a summary of it? I would think flatten could go in itertools,
but clearly there must some reason why its not there. I imagine the
duplication with it.chain.from_iter + "There should be one-- and preferably
only one --obvious way to do it."? As for what it should do, I'm guessing
the controversy was over flattening one level vs all levels? That makes
sense and is good to know. I guess I won't pick `flatten` as one of my
first functions to pick for a writeup. On a similar note, do you (or anyone
else) have an intuition for which of these functions --- judging by name
only (so you don't have to click any links) --- might be the least
controversial? I'm not very good at judging controversy, which is one of
the main reasons for this initial email.

Maybe `expandpath` to os.path? Or perhaps start with ub.modname_to_modpath
and
ub.modpath_to_modname to importlib? Maybe some of the dict-methods? Perhaps
I'm overestimating the clear usefulness of any of these functions to the
stdlib?

On Sat, Nov 10, 2018 at 9:14 PM Steven D'Aprano  wrote:

> On Sat, Nov 10, 2018 at 08:36:52PM -0500, Jonathan Crall wrote:
> > I'm interested in proposing several additions to the Python standard
> > library, and I would like more information on the procedure for doing so.
> > Are all additions done via a PEP?
>
> Not necessarily. Small, obvious enhancements can go straight to the
> bug tracker. The tricky part is deciding what is "obvious".
>
> Sometimes there's a good, useful function than doesn't get added because
> there's no reasonable place to put it. For example, a "flatten" function
> has been talked about since Python 1.x days, and we still don't have a
> standard solution for it, because (1) it isn't clear *precisely* what it
> should do, and (2) it isn't clear where it should go.
>
> Given that once something gets added to the std lib, it is hard to
> remove it or even rename it, its better to be conservative about adding
> things and leave it to third party libraries to cover the gaps.
>
>
> > If not what is the procedure. If so, I've
> > read that the first step was to email this board and get feedback.
>
> That's a good idea. If the enhancement request isn't both small and
> obvious, or is the least bit controversial, you'll usually be sent back
> here.
>
>
> > I have a library called `ubelt` that contains several tools that I think
> > might be worthy of adding to the standard library.
>
> Generally speaking, we don't typically add grab-bags of random utility
> functions. There is no "utilities" or "toolbox" module in the std lib.
>
>
> [...]
> > Here is a tentative list of interesting functions. Hopefully the names
> are
> > descriptive (if not, see docstrings: https://github.com/Erotemic/ubelt)
>
> Sorry, some of these aren't descriptive enough, and if you're trying to
> make a pitch for these features, you ought to give at least a
> one-sentence explanation of them here in the email. You will lose half
> your audience as soon as you ask them to click through to a link, and
> even if they do, that risks splitting the discussion across two places.
>
> My advice is to collate the functions you want to add into groups of
> related functionality, find the class or module in the std lib where you
> think they belong, and begin a new thread for each group. E.g. "New dict
> methods", "New importlib functions".
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Jon
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposing additions to the standard library

[Jonathan Crall]

@Stéfane Bolton looks really neat! I'll take a look. Some of my stuff may
fit better as a PR for this library.

Also I don't think its foolish to depend on a package for one function,
given that that (a) that function is really useful or (b) the size of the
dependency itself is small. Given my initial impressions of boltons, I
would guess that it doesn't have a large download size of runtime impact.
Although if this case keeps reoccurring with a particular function, then
perhaps that function might improve the stdlib? After really reviewing my
stuff, I think a few functions I have would make the stdlib better, but its
probably only a small few. I bet there are things in bolton that would
improve the stdlib as well, but I do agree that "semi-standard" libraries
(e.g. numpy / scipy / what-I-hope-ubelt-to-be) might be better place for
these functions that would otherwise cause costly-clutter in the stdlib.

On Tue, Nov 13, 2018 at 2:05 AM Stéfane Fermigier  wrote:

> Are you aware of https://boltons.readthedocs.io/ (whose motto is
> "Functionality that should be in the standard library.") ?
>
> Or similar endeavours such as:
>
> - https://pypi.org/project/auxlib/
> - https://pypi.org/project/omakase/
> - (And probably many others on PyPI with similar descriptions such as "a
> library of stuff I'm using / we're using at company X for all my / our
> project(s)...")
>
> Or the functional libraries listed here:
> https://github.com/sfermigier/awesome-functional-python/blob/master/README.md#libraries
>
> => IMHO there is room for a "semi-standard" library, stuff that's not
> included by default and has a release lifecycle more active that Python
> itself, but that can be considered the standard by a large group of users.
>
> Similar ideas can be found for instance in Java with Apache Commons (
> https://commons.apache.org/ -> "an Apache project focused on all aspects
> of reusable Java components."). One could argue, though, that the Java
> standard library is much less developed than the Python standard library,
> so it's much easier to justify the existence of Apache Commons than a
> similar Python project.
>
> There is also the question of the porosity between such a project and the
> stdlib, which is the essence of the original question by the OP.
>
> Another interesting issue is the granularity of such a project. I
> sometimes, and somewhat foolishly, make libraries such as toolz or boltons
> a dependency of my projects, for just one or two function calls from my
> code.
>
> Regards,
>
>   S.
>
>
> On Sun, Nov 11, 2018 at 2:37 AM Jonathan Crall  wrote:
>
>> I'm interested in proposing several additions to the Python standard
>> library, and I would like more information on the procedure for doing so.
>> Are all additions done via a PEP? If not what is the procedure. If so, I've
>> read that the first step was to email this board and get feedback.
>>
>> I have a library called `ubelt` that contains several tools that I think
>> might be worthy of adding to the standard library.
>>
>> Here's my bullet point pitch:
>>
>>- Python is batteries included. Ubelt contains extra batteries.
>>function are extra batteries.
>>- Most function in ubelt are fast. All 222 tests takes 7.33 seconds.
>>- Ubelt has 100% test coverage (sans `# nocover` locations).
>>- I'm only championing a subset of the functions in ubelt. There are
>>certainly functions in there that do not belong in the standard library.
>>- I have a Jupyter notebook that give a demo of some select functions
>>(not necessarily the same as the ones proposed here):
>>
>> https://github.com/Erotemic/ubelt/blob/master/docs/notebooks/Ubelt%20Demo.ipynb
>>- I do have documentation (mostly in docstrings) and in the docs
>>folder, but I've been having trouble auto-updating read-the-docs. Here is
>>the link anyway: https://ubelt.readthedocs.io/en/latest/
>>
>> Here is a tentative list of interesting functions. Hopefully the names
>> are descriptive (if not, see docstrings:
>> https://github.com/Erotemic/ubelt)
>>
>> ub.cmd
>>
>> ub.compressuser
>>
>> ub.group_items
>>
>> ub.dict_hist
>>
>> ub.find_duplicates
>>
>> ub.AutoDict
>>
>> ub.import_module_from_path
>>
>> ub.import_module_from_name
>>
>> ub.modname_to_modpath,
>>
>> ub.modpath_to_modname
>>
>> ub.ProgIter
>>
>> ub.ensuredir
>>
>> ub.expandpath
>>
>>
>> almost everything in util_list:
>>
>> allsame, argmax, argmin, argsort, argunique,
>>

[Python-ideas] Re: SerialExecutor for concurrent.futures + Convenience constructor

[Jonathan Crall]

Based on the conversation so far, I agree with @Kyle Stanley's breakdown of
the proposal. I think shelving the "*Add a new way to create and specify
executor*" and focusing on "*Add a SerialExecutor, which does not use
threads or processes*" is the best way forward.

For context, I'm a machine learning researcher and developer. I've made
extensive use of both thread and process based parallelism (and I'm very
much looking forward to subinterpreters).  I use threads for tasks like
downloading files, running background tasks when my GPU computations are
the bottleneck, and other IO related tasks. I use processes for image
processing and other CPU bound tasks.

@Andrew Barnert 's analysis of the use case is spot on.
Andrew states:

>
> I’m pretty sure what he meant is that the developer _usually_ wants the
> task to run in parallel, but in some specific situation he wants it to
> _not_ run in parallel.

The concrete use case I’ve run into is this: I’ve got some parallel code
> that has a bug. I’m pretty sure the bug isn’t actually related to the
> shared data or the parallelism itself, but I want to be sure. I replace the
> ThreadPoolExecutor with a SyncExecutor and change nothing else about the
> code, and the bug still happens. Now I’ve proven that the bug isn’t related
> to parallelism. And, as a bonus, I’ve got nice logs that aren’t interleaved
> into a big mess, so it’s easier to track down the problem.


This is exactly the use case that I run into, but this isn't the only use
case for SerialExecutor. @Antoine Pitrou put it nicely:

Being able to swap a ThreadPoolExecutor or ProcessPoolExecutor with a
> serial version using the same API can have benefits in various
> situations.  One is easier debugging (in case the problem you have to
> debug isn't a race condition, of course :-)).  Another is writing a
> library a command-line tool or library where the final decision of
> whether to parallelize execution (e.g. through a command-line option for
> a CLI tool) is up to the user, not the library developer.


Antoine's second point is  important in certain multiuser or limited
hardware environments. On my personal machine I use all the compute
available, but on a shared system I need to constrain the resources I'm
using. Disabling parallelism can also be useful on hardware like the
raspberry pi.


1) Debugging parallel code: this is the use case stated by @Andrew Barnert
. Serial code is easier to debug, and currently the
executor API requires restructuring of the code if you want to rule out
parallelism as the source of a bug.
2) Some programs run better on one CPU in certain hardware / multiuser
environments : depending on the hardware you may want to disable
parallelism in your code. Many times I check for a `--serial` flag in the
command line to disable parallelism.

This proposal isn't so much about faking parallelism as it is disabling it
when you need to. If you set `max_workers` to 0 in ThreadPoolExecutor or
ProcessPoolExecutor you get an error. I don't think that disabling
parallelism is an uncommon use case. As previously mentioned it has uses in
debugging and allowing the user to control the flow of execution. This
second case is useful when your parallel code has a race condition that
doesn't appear on your machine, but it does on your customer's machine. The
current futures API does not work if you need to fallback on
single-threaded execution, which means that if the developer wants the
option to disable parallelism they have to maintain two different
implementations of the same functionality. A serial executor would allow
duck-typing to solve that problem.


Also, as a sidenote, I much more prefer the term "SyncExecutor" rather than
> "SerialExecutor". I think the former is a bit more clear at defining it's
> actual purpose.


FWIW I found the term "SyncExecutor" really confusing when I was reading
this thread. I thought it was short for Synchonized, but I just realized
its actually short for Synchronous, which makes much more sense. While
SynchronousExecutor makes more sense to me, it is also more verbose and
difficult to spell.

It seems there are two possible design decisions for a serial executor:
> - one is to execute the task immediately on `submit()`
> - another is to execute the task lazily on `result()`

This could for example be controlled by a constructor argument to
> SerialExecutor.


This is a great idea. I think I like the default being lazy execution, but
giving the user control over that would increase the usefulness.

I also see some conversation about a public API to query and get the state
of a process. That's likely because my implementation abuses a private
member variable, but I think it might be possible to implement
"SerialExecutor" without exposing state setter / getters. I think @Kyle
Stanley's idea makes sense:

 ``submit()`` could potentially "fake" the process of scheduling the
> execution of the function, but without directly executing it; perhaps with
> 

[Python-ideas] SerialExecutor for concurrent.futures + Convenience constructor

[Jonathan Crall]

I'd like to propose an improvement to `concurrent.futures`. The library's
ThreadPoolExecutor and ProcessPoolExecutor are excellent tools, but there
is currently no mechanism for configuring which type of executor you want.
Also, there is no duck-typed class that behaves like an executor, but does
its processing in serial. Often times a develop will want to run a task in
parallel, but depending on the environment they may want to disable
threading or process execution. To address this I use a utility called a
`SerialExecutor` which shares an API with
ThreadPoolExecutor/ProcessPoolExecutor but executes processes sequentially
in the same python thread:


```python
import concurrent.futures

class SerialFuture( concurrent.futures.Future):
"""
Non-threading / multiprocessing version of future for drop in
compatibility
with concurrent.futures.
"""
def __init__(self, func, *args, **kw):
super(SerialFuture, self).__init__()
self.func = func
self.args = args
self.kw = kw
# self._condition = FakeCondition()
self._run_count = 0
# fake being finished to cause __get_result to be called
self._state = concurrent.futures._base.FINISHED

def _run(self):
result = self.func(*self.args, **self.kw)
self.set_result(result)
self._run_count += 1

def set_result(self, result):
"""
Overrides the implementation to revert to pre python3.8 behavior
"""
with self._condition:
self._result = result
self._state = concurrent.futures._base.FINISHED
for waiter in self._waiters:
waiter.add_result(self)
self._condition.notify_all()
self._invoke_callbacks()

def _Future__get_result(self):
# overrides private __getresult method
if not self._run_count:
self._run()
return self._result


class SerialExecutor(object):
"""
Implements the concurrent.futures API around a single-threaded backend

Example:
>>> with SerialExecutor() as executor:
>>> futures = []
>>> for i in range(100):
>>> f = executor.submit(lambda x: x + 1, i)
>>> futures.append(f)
>>> for f in concurrent.futures.as_completed(futures):
>>> assert f.result() > 0
>>> for i, f in enumerate(futures):
>>> assert i + 1 == f.result()
"""
def __enter__(self):
return self

def __exit__(self, ex_type, ex_value, tb):
pass

def submit(self, func, *args, **kw):
return SerialFuture(func, *args, **kw)

def shutdown(self):
pass

```

In order to make it easy to choose the type of parallel (or serial) backend
with minimal code changes I use the following "Executor" wrapper class
(although if this was integrated into concurrent.futures the name would
need to change to something better):

```python
class Executor(object):
"""
Wrapper around a specific executor.

Abstracts Serial, Thread, and Process Executor via arguments.

Args:
mode (str, default='thread'): either thread, serial, or process
max_workers (int, default=0): number of workers. If 0, serial is
forced.
"""

def __init__(self, mode='thread', max_workers=0):
from concurrent import futures
if mode == 'serial' or max_workers == 0:
backend = SerialExecutor()
elif mode == 'thread':
backend = futures.ThreadPoolExecutor(max_workers=max_workers)
elif mode == 'process':
backend = futures.ProcessPoolExecutor(max_workers=max_workers)
else:
raise KeyError(mode)
self.backend = backend

def __enter__(self):
return self.backend.__enter__()

def __exit__(self, ex_type, ex_value, tb):
return self.backend.__exit__(ex_type, ex_value, tb)

def submit(self, func, *args, **kw):
return self.backend.submit(func, *args, **kw)

def shutdown(self):
return self.backend.shutdown()
```

So in summary, I'm proposing to add a SerialExecutor and SerialFuture class
as an alternative to the ThreadPool / ProcessPool executors, and I'm also
advocating for some sort of "ParamatrizedExecutor", where the user can
construct it in "thread", "process", or "serial" model.

--
-Jon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LMTQ2AI6A7UXEFVHRGHKWD33H24FGM6G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: SerialExecutor for concurrent.futures + Convenience constructor

[Jonathan Crall]

This implementation is a proof-of-concept that I've been using for awhile
<https://gitlab.kitware.com/computer-vision/ndsampler/blob/master/ndsampler/util_futures.py>.
Its certain that any version that made it into the stdlib would have to be
more carefully designed than the implementation I threw together. However,
my implementation demonstrates the concept and there are reasons for the
choices I made.

First, the choice to create a SerialFuture object that inherits from the
base Future was because I only wanted a process to run if the
SerialFuture.result method was called. The most obvious way to do that was
to overload the `result` method to execute the function when called.
Perhaps there is a better way, but in an effort to KISS I just went with
the <100 line version that seemed to work well enough.

The `set_result` is overloaded because in Python 3.8, the base
Future.set_result function asserts that the _state is not FINISHED when it
is called. In my proof-of-concept implementation I had to set state of the
SerialFuture._state to FINISHED in order for `as_completed` to yield it.
Again, there may be a better way to do this, but I don't claim to know what
that is yet.

I was thinking that a factory function might be a good idea, but if I was
designing the system I would have put that in the abstract Executor class.
Maybe something like


```
@classmethod
def create(cls, mode, max_workers=0):
""" Create an instance of a serial, thread, or process-based executor
"""
from concurrent import futures
if mode == 'serial' or max_workers == 0:
return futures.SerialExecutor()
elif mode == 'thread':
return futures.ThreadPoolExecutor(max_workers=max_workers)
elif mode == 'process':
return futures.ProcessPoolExecutor(max_workers=max_workers)
else:
raise KeyError(mode)
```

I do think that it would improve the standard lib to have something like
this --- again perhaps not this exact version (it does seem a bit weird to
give this method to an abstract class), but some common API that makes it
easy for the user to swap between the backend Executor implementation. Even
though the implementation is "trivial", lots of things in the standard lib
are, but they the reduce boilerplate that developers would otherwise need,
provide examples of good practices to new developers, and provide a defacto
way to do something that might otherwise be implemented differently by
different people, so it adds value to the stdlib.

That being said, while I will advocate for the inclusion of such a factory
method or wrapper class, it would only be a minor annoyance to not have it.
On the other hand I think a SerialExecutor is something that is sorely
missing from the standard library.

On Sat, Feb 15, 2020 at 5:16 PM Andrew Barnert  wrote:

> > On Feb 15, 2020, at 13:36, Jonathan Crall  wrote:
> >
> > Also, there is no duck-typed class that behaves like an executor, but
> does its processing in serial. Often times a develop will want to run a
> task in parallel, but depending on the environment they may want to disable
> threading or process execution. To address this I use a utility called a
> `SerialExecutor` which shares an API with
> ThreadPoolExecutor/ProcessPoolExecutor but executes processes sequentially
> in the same python thread:
>
> This makes sense. I think most futures-and-executors frameworks in other
> languages have a serial/synchronous/immediate/blocking executor just like
> this. (And the ones that don’t, it’s usually because they have a different
> way to specify the same functionality—e.g., in C++, you only use executors
> via the std::async function, and you can just pass a launch option instead
> of an executor to run synchronously.)
>
> And I’ve wanted this, and even built it myself at least once—it’s a great
> way to get all of the logging in order to make things easier to debug, for
> example.
>
> However, I think you may have overengineered this.
>
> Why can’t you use the existing Future type as-is? Yes, there’s a bit of
> unnecessary overhead, but your reimplementation seems to add almost the
> same unnecessary overhead. And does it make enough difference in practice
> to be worth worrying about anyway? (It doesn’t for my uses, but maybe
> you’re are different.)
>
> Also, why are you overriding set_result to restore pre-3.8 behavior? The
> relevant change here seems to be the one where 3.8 prevents executors from
> finishing already-finished (or canceled) futures; why does your executor
> need that?
>
> Finally, why do you need a wrapper class that constructs one of the three
> types at initialization and then just delegates all methods to it? Why not
> just use a factory function that constructs and returns an instance of one
> of the three types directly? And, given how trivial that f

[Python-ideas] Re: more readable "if" for multiple "in" condition

[Jonathan Crall]

For what its worth, I find the proposed syntax confusing and difficult to
read. Conceptually it makes sense, but using existing set operations seems
like a far more readable way to handle this case.

On Tue, Dec 31, 2019 at 11:03 AM Richard Damon 
wrote:

> On 12/31/19 1:20 AM, iman.h.a.kha...@gmail.com wrote:
> > Hi
> > I think the following syntax's:
> >
> > if foo in foobar or bar in foobar or baz in foobar:
> >  pass
> > if foo in foobar and bar in foobar and baz in foobar:
> >  pass
> >
> > can be more readable and shorter if written as:
> >
> > if foo or bar or baz in foobar:
> >  pass
> > if foo and bar and baz in foobar:
> >  pass
> >
> > maybe for "is" instead of:
> >
> > if foobar is foo or foobar is bar or foobar is baz:
> >  pass
> >
> > if foobar is foo or bar or baz:
> >  pass
> >
> > now the recommended syntax translate to this: (I think so)
> >
> > if foo (IS NOT '0' or None or empty) or bar (IS NOT '0' or None or
> empty) or baz in foobar
> >
> > so probably we must introduce a new type of 'or' (I recommend the
> CAPITAL "OR","AND") so this:
> >
> > if foo OR bar OR baz in foobar:
> >  pass
> >
> > translates to this:
> >
> > if foo in foobar or bar in foobar or baz in foobar:
> >  pass
>
> IF I were to change the syntax ( which I don't propose), I believe the
> construct like foo or bar or baz in foobar to give you issues parsing.
> An alternative which is, in my opinion, more readable would be if
> any(foo, bar, baz) in foobar (and the alternative if all(foo, bar, baz)
> in foobar). To do this any and all could make a specially tagged tupl
> which interacts with operators like 'in' in a special way where it
> applies each of its members and or/ands the results.
>
> It may even be possible to do this with the existing Python by defining
> a suitable class any and all with an appropriate constructor and
> implementing its own version of things like in. (not sure if we can
> override 'is')
>
> --
> Richard Damon
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/VGN4PW5LV7LMDA3ORP33BYBHJDEFMEGU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Jon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OTKTGKG4Y7ZTZ4VCUY3VCQCJGFNFVNJP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: 'Infinity' constant in Python

[Jonathan Crall]

Given that this is unlikely to see any change in core Python, perhaps you
can get a decent solution with a third party library? If your main issue is
that repr doesn't provide an "eval-able" string, then may I suggest using
the "most-used" function in the ubelt utility library: ubelt.repr2

Two main goals of repr2 are to provide nice string representations of nested
data structures and make those "eval-able" whenever possible. As an example
take the value `float('inf')`, which normaly has a non-evalable repr of
`inf`:

>>> !pip install ubelt
>>> import ubelt as ub
>>> ub.repr2(float('inf'))
"float('inf')"

The `newline` (or `nl`) keyword argument can control how deep in the nesting
newlines are allowed.

>>> print(ub.repr2({1: float('nan'), 2: float('inf'), 3: 3.0}))
{
1: float('nan'),
2: float('inf'),
3: 3.0,
}

>>> print(ub.repr2({1: float('nan'), 2: float('inf'), 3: 3.0}, nl=0))
{1: float('nan'), 2: float('inf'), 3: 3.0}


You can also define or overwrite how representations for different types are
created. You can either create your own extension object, or you can
monkey-patch `ub.util_format._FORMATTER_EXTENSIONS` without specifying the
extensions keyword argument (although this will be a global change).

>>> extensions = ub.FormatterExtensions()
>>> @extensions.register(float)
>>> def my_float_formater(data, **kw):
>>> return "monkey({})".format(data)
>>> print(ub.repr2({1: float('nan'), 2: float('inf'), 3: 3.0}, nl=0,
extensions=extensions))
{1: monkey(nan), 2: monkey(inf), 3: monkey(3.0)}



On Fri, Sep 4, 2020 at 12:49 PM Cade Brown  wrote:

> I am positing that Python should contain a constant (similar to True,
> False, None), called Infinity.
>
> It would be equivalent to `float('inf')`, i.e. a floating point value
> representing a non-fininte value. It would be the positive constant;
> negative infinity could retrieved via `-Infinity`
>
> Or, to keep float representation the same, the name `inf` could be used,
> but that does not fit Python's normal choice for such identifiers (but
> indeed, this is what C uses which is the desired behavior of string
> conversion)
>
> I think there are a number of good reasons for this constant. For example:
>   * It is also a fundamental constant (similar to True, False, and None),
> and should be representable as such in the language
>   * Requiring a cast from float to string is messy, and also obviously
> less efficient (but this performance difference is likely insignificant)
>   * Further, having a function call for something that should be a
> constant is a code-smell; in general str -> float conversion may throw an
> error or anything else and I'd rather not worry about that.
>   * It would make the useful property that `eval(repr(x)) == x` for
> floating point numbers (currently, `NameError: name 'inf' is not defined`)
>
> This makes it difficult to, for example, naively serialize a list of
> floats. For example:
>
> ```
> >>> x = [1, 2, 3, 4]
> >>> repr(x)
> '[1, 2, 3, 4]'
> >>> eval(repr(x)) == x
> True
> >>> x = [1, 2, 3, float('inf')]
> >>> repr(x)
> '[1, 2, 3, inf]'
> >>> eval(repr(x)) == x
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 1, in 
> NameError: name 'inf' is not defined
> ```
>
> To me, this is problematic; I would expect it to work seamlessly as it
> does with other floating point constants.
>
> A few rebuttals/claims against:
>   - Creating a new constant (Infinity) which is unassignable may break
> existing code
>   - Converting a float to string is not the same as it is in C. Whil
>
> I also realize that there is `math.inf`, but I argue that the constant is
> more fundamental than that, and it still doesn't solve the problem with
> `repr()` I described
>
> Thanks,
> 
> *Cade Brown*
> Research Assistant @ ICL (Innovative Computing Laboratory)
> Personal Email: brown.c...@gmail.com
> ICL/College Email: c...@utk.edu
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/XMA6KOBLPABV7EL5GV2BIRC2ESYKXMVV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/J74OUML4XMEOHSSG5OXK3K45VOICYWUS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New feature

[Jonathan Crall]

I just want to point out that I can think of a valid use case for `clf`.
I'm not sure if it was mentioned.

In the case where you have a script that produces a lot of output, a common
task might be scrolling to the beginning to check an output. If your screen
was not fresh, and you had a lot of previous output (say from running the
script multiple times), then it is hard to find the beginning unless you
have previously cleared the screen.

If you have cls, you can ensure that your big crazy
undergrad-programming-level script always clears the screen when it runs.
Maybe that helps you get your homework in on time. There is a task you
commonly do (clear the screen via cls or clear) programatically in python.

You may say: why are you not using logging? Well, perhaps --- because it
seems it is typically students who want to do this --- perhaps they don't
know how to do logging, or have doubts that they know how to do it
properly.

It seems that something like this might be better as a third party library,
given that it's more of a stdout + console specific thing. But on the other
hand this isn't much worse than `os.startfile` on windows.



On Fri, Oct 16, 2020 at 1:21 PM David Mertz  wrote:

> I agree with Guido here.  Although I really don't care about the
> capability myself, it feels like enough people do want a "clear screen"
> function... and from the discussion, in feels like there are a LOT of
> variations in how to do it across different operating systems, OS versions,
> terminals, shells, etc.
>
> Having a common interface of `os.clear()` that did whatever funny thing a
> particular environment needed would save some folks trouble.  Of course,
> I'm not certain how far it is possible to auto-detect the environment
> details within that function to "do the right thing" ... but probably there
> are clever hacks that get to 90% working.
>
> On Fri, Oct 16, 2020 at 1:03 PM Rob Cliffe via Python-ideas <
> python-ideas@python.org> wrote:
>
>>
>>
>> On 16/10/2020 13:55, Chris Angelico wrote:
>> > On Fri, Oct 16, 2020 at 11:08 PM Rob Cliffe 
>> wrote:
>> >>
>> >>
>> >> On 16/10/2020 11:59, Chris Angelico wrote:
>> >>> On Fri, Oct 16, 2020 at 8:21 PM Rob Cliffe via Python-ideas
>> >>>  wrote:
>> 
>>  On 13/10/2020 23:35, Guido van Rossum wrote:
>> > Can one of the educators on the list explain why this is such a
>> > commonly required feature? I literally never feel the need to clear
>> my
>> > screen -- but I've seen this requested quite a few times in various
>> > forms, often as a bug report "IDLE does not support CLS". I presume
>> > that this is a common thing in other programming environments for
>> > beginners -- even C++ (given that it was mentioned). Maybe it's a
>> > thing that command-line users on Windows are told to do frequently?
>> > What am I missing that students want to do frequently? Is it a
>> > holdover from the DOS age?
>> >
>>  Sometimes I want a program that displays (more than 1 line of)
>> real-time
>>  information in a Windows CMD box and refreshes it every few seconds
>>  (e.g. progress displays, monitoring open
>>  files/locks/connections/downloads etc.).  It is natural to clear the
>>  screen and display the updated information.
>> >>> Natural perhaps, but ugly. Much better to reposition the cursor and
>> >>> overwrite the previous text, with "clear to end of line" as required;
>> >>> that way, you avoid flicker.
>> >>>
>> >>> C
>> >> I do precisely that in many of my programs for e.g. single-line
>> progress
>> >> displays.
>> >> But for multi-line output I don't know of any way to move the cursor
>> >> back up.
>> >> I work in Windows 10.
>> > Try \x1b[A to move up a line, should work.
>> >
>> > ChrisA
>> Thanks Chris, but no luck.  It just echoes it, with the \x1b (Escape)
>> echoed as a character that looks like a question mark inside a box.
>> Earlier I did try googling for ways of moving the cursor, but almost all
>> I found was ways of moving the *mouse* cursor, and the rest was
>> irrelevant.
>> Rob Cliffe
>> > ___
>> > Python-ideas mailing list -- python-ideas@python.org
>> > To unsubscribe send an email to python-ideas-le...@python.org
>> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> > Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/5VZGBRIEW5UPQ6VCYQXC3UUNNHQXNFRB/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/5SBLJZAT3CH35ILXLSCLX5R62BQX3FQE/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> The dead increasingly dominate and 

[Python-ideas] Re: New feature

[Jonathan Crall]

@Paul Moor, I agree with you. Personally, I'd rather not use it, and if it
was used in an importable module I'd claim it was bad practice.

But in general I don't think there is a right answer of if you *should *or
not. I think it is valid for personal preferences to vary here. If this is
something someone forgets to do a lot (and everyone has their own personal
quirks about what's hard/easy for them to remember), then it could be
annoying to the point where this feature could really help them.

But yes, in most cases I'd rather not use it, and I would get peeved if
someone's code they advertised as reusable used this (but then again I also
get peeved when people execute print statements at import time).



On Fri, Oct 16, 2020 at 5:02 PM Paul Moore  wrote:

> On Fri, 16 Oct 2020 at 20:43, Jonathan Crall  wrote:
> >
> > @Chris Angelico If you are more application focused, or trying to get a
> scientific result, and you don't care too much about the reusability of
> your scripts, then I can see the validity of this use case. But I would not
> want to install a module on pypi that made use of this. From a software
> perspective this is 100% bad practice, but from a practitioner perspective
> --- and there are a lot of practitioners that only use Python as a tool
> (like a DSL) --- I can see wanting it.
> >
> > So, that's the question: does the Python stdlib want to let you do this?
> Will the number of software engineers who use this incorrectly outweigh the
> number of people who seem to want this for a valid reason? I'm not sure
> what the answer is.
> >
> > FWIW executing `print('\x1b[H\x1b[2J\x1b[3J')` in my IPython terminal
> cleared the screen.
>
> Why would you want to put this in your script, so that you have no
> choice (short of editing the script) as to whether it happens? Surely
> better to clear your terminal from the command prompt *before* running
> your script? Yes, you can forget, but I'd rather forget occasionally
> than lose essential data because I ran a script that I'd forgotten has
> an os.clear() at the top...
>
> I suspect Guido's right, it's asked for often enough, and fiddly
> enough to write, that adding it is a net gain. But I'd be very
> unlikely to use it myself, and I'd strongly advise people against
> using it if asked.
>
> Paul
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L5UWOFF27IC4LBCQZ4SFQ26RLLDDNBPJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New feature

[Jonathan Crall]

@Chris Angelico  If you are more application focused, or
trying to get a scientific result, and you don't care too much about the
reusability of your scripts, then I can see the validity of this use case.
But I would not want to install a module on pypi that made use of this.
>From a software perspective this is 100% bad practice, but from a
practitioner perspective --- and there are a lot of practitioners that only
use Python as a tool (like a DSL) --- I can see wanting it.

So, that's the question: does the Python stdlib want to let you do this?
Will the number of software engineers who use this incorrectly outweigh the
number of people who seem to want this for a valid reason? I'm not sure
what the answer is.

FWIW executing `print('\x1b[H\x1b[2J\x1b[3J')` in my IPython terminal
cleared the screen.

On Fri, Oct 16, 2020 at 2:45 PM Chris Angelico  wrote:

> On Sat, Oct 17, 2020 at 5:40 AM Jonathan Crall  wrote:
> >
> > I just want to point out that I can think of a valid use case for `clf`.
> I'm not sure if it was mentioned.
> >
> > In the case where you have a script that produces a lot of output, a
> common task might be scrolling to the beginning to check an output. If your
> screen was not fresh, and you had a lot of previous output (say from
> running the script multiple times), then it is hard to find the beginning
> unless you have previously cleared the screen.
> >
>
> This is exactly why I think this should NOT be made too easy. That's a
> perfect example of an attractive nuisance: instead of creating a
> divider (say, a half a dozen blank lines, or a row of hyphens with a
> blank or two each side), you're throwing away all information from the
> previous output, making it impossible to compare the two. Once your
> script has a clear-screen at the start of it, you lose the option to
> examine history, whereas a simple few blank lines would have a similar
> effect without losing anything.
>
> If it is added to the stdlib, I hope that it's buried away where
> people find it only if they're actually looking for it. There ARE
> valid use-cases, but there are far far more cases where it shouldn't
> be used.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/7BZX6GHHWZ4IIP5ITRYCRKKHDGCM3OVS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HCHYR6ZXSRUWAMSJUG2HDZCVGHCT7MJ6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[Jonathan Crall]

FWIW I'm +1 on `d.keys()[i]`,  `d.values()[i]`,  `d.items()[i]`. I don't
see any major issue with adding a __getitem__ to the view classes. I think
it's a great idea, and it would make my life easier and would replace the
primary use case where I currently use `ubelt.peek
`.

On Mon, Jun 29, 2020 at 2:57 PM Stestagg  wrote:

> I'm quite supportive (+1) of the proposal to add numeric indexing to the
> 'dict_*' views.
>
> Given that dictionaries are now ordered, it seems reasonable to look-up,
> for example, keys by index,
>
> One simple example of where this is surprising is in the following:
>
> >>> random.choice({'a': 1,}.keys())
> TypeError: 'dict_keys' object is not subscriptable
>
> This could be 'fixed' with a simple hack in the `random.choice` code, bit
> equally, implementing __getitem__ on `dict_*` types would enrich the
> interface.
>
> Several times now, I've had the need to 'just get any key/value' from a
> large dictionary.  I usually try first to run `var.keys()[0]` only to be
> told that I'm not allowed to do this, and instead have to ask python to
> make a copy of this datastructure with a different type, just so I can
> perform the index operation.  This is possible, but seems redundant, and
> reinforces bad practices around creating copies of potentially large
> structures.
>
> Another use-cases is doing variations of reduce() over dictionaries, where
> getting an initial value from the dict, and then performing operations over
> the remaining items is much simpler to do with indexing on the views.
>
> Steve
>
>
>
>
>
>
> On Mon, Jun 29, 2020 at 1:27 PM Hans Ginzel  wrote:
>
>> Thank you.
>>
>> On Fri, Jun 26, 2020 at 02:50:22PM -0300, Joao S. O. Bueno wrote:
>> >On Fri, 26 Jun 2020 at 14:30, Hans Ginzel  wrote:
>> >> thank you for making dict ordered.
>> >> Is it planned to access key,value pair(s) by index? See
>> >> https://stackoverflow.com/a/44687752/2556118 for example. Both for
>> >> reading and (re)writing?
>> >> Is it planned to insert pair(s) on exact index? Or generally to slice?
>> See
>> >> splice() in Perl, https://perldoc.perl.org/functions/splice.html.
>> >> …
>> >>
>> >These are odd requirements.
>> >
>> >No - Python dictionaries are ordered, by order of insertion only, but one
>> >can't generally do any manipulation by the numeric index of
>> >a dictionary entry - and it will stay that way.
>>
>> That is fully corret to respect the _insertion_ order.
>>
>> >If you need such an hybrid data structure, you could just have
>> >a list of tuples as data structure, and use
>> collections.abc.MutableMapping
>> >to provide a dict-like interface to it (an index for better than linear
>> search).
>> >
>> >I could create such a data structure if you want,
>>
>> Thank you, I will write it myself.
>> H.
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/BWHRHYFTPJVHXKER5OUKARBS3N3OCSNK/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/32DNWTZBBD6XXMPIPRGCKBMA26M7VPFL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T4LVK377JOXJETFJOOGYFRO7F5KXKNSP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Please consider adding numbers.Boolean

[Jonathan Crall]

I disagree that doing arithmetic on boolean variables doesn't make sense.
Indicator variables which take a value of either zero are one are extremely
common. Dirac or Kronecker delta functions are also examples of functions
where it arithmetically makes sense to talk about values in the boolean
domain. Of course that does mean that you define True as 1 and False as 0,
but that is a very standard convention and it is the case in Python.

I don't have a strong opinion on the proposal itself, but saying
"arithmetic on booleans doesn't doesn't make mathematical sense" is False.

On Mon, Jun 22, 2020 at 5:08 AM Greg Ewing 
wrote:

> On 22/06/20 12:19 pm, Neil Girdhar wrote:
> > I'm just curious if there was a reason why Boolean was omitted from the
> > numeric tower in the numbers library?
>
> Python's bool type is a subclass of int for historical reasons,
> not because it's conceptually a numeric type. Doing arithmetic on
> it doesn't really make mathematical sense.
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/CMDSMDVCT5ABSLILV3U772QDLI2IUCAZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WZTFFH4BHZAS4YSKVZOSS6JTA3VYFIDZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bringing the print statement back

[Jonathan Crall]

I wouldn't mind if this *only *worked for the specific characters "print".

However, I think that allowing developers to call complex methods without
parentheses will lead to very difficult to read code.I think that
surrounding arguments with non-whitespace buffer characters is important
for readability. It makes things feel contained. For the print function it
seems less bad because there is only one argument and oftentimes it is
surrounded by quotes, which mitigates the lack of a buffer character. I
dread reading someone else's code that includes heavy usage of this pattern
for generic function calls.

In summary: "boo, hiss". For generic method calls I think this is a
terrible idea. However, I would seriously consider a special case grammar
change that only allows functions (or variables?) named "print" to be
called this way. I think this has benefits for backwards-compatibility,
although I will still personally use parens around my prints (because being
surrounded by parens looks so freakin' good) it wouldn't make me tear out
my hair out to see: `print x` in code reviews.

On Tue, Jun 9, 2020 at 8:09 PM Guido van Rossum  wrote:

> In Python 3.10 we will no longer be burdened by the old parser (though 3rd
> party tooling needs to catch up).
>
> One thing that the PEG parser makes possible in about 20 lines of code is
> something not entirely different from the old print statement. I have a
> prototype:
>
> Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun  9 2020,
> 16:31:17)
> [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> Cannot read termcap database;
> using dumb terminal settings.
> >>> print 2+2
> 4
> >>> print "hello world"
> hello world
> >>> print "hello", input("Name:")
> Name:Guido
> hello Guido
> >>> print 1, 2, 3, sep=", "
> 1, 2, 3
> >>>
>
> But wait, there's more! The same syntax will make it possible to call
> *any* function:
>
> >>> len "abc"
> 3
> >>>
>
> Or any method:
>
> >>> import sys
> >>> sys.getrefcount "abc"
> 24
> >>>
>
> Really, *any* method:
>
> >>> class C:
> ...   def foo(self, arg): print arg
> ...
> >>> C().foo 2+2
> 4
> >>>
>
> There are downsides too, though. For example, you can't call a method
> without arguments:
>
> >>> print
> 
> >>>
>
> Worse, the first argument cannot start with a parenthesis or bracket:
>
> >>> print (1, 2, 3)
> 1 2 3
> >>> C().foo (1, 2, 3)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: C.foo() takes 2 positional arguments but 4 were given
> >>> print (2+2), 42
> 4
> (None, 42)
> >>> C().foo [0]
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: 'method' object is not subscriptable
> >>>
>
> No, it's not April 1st. I am seriously proposing this (but I'll withdraw
> it if the response is a resounding "boo, hiss"). After all, we currently
> have a bunch of complexity in the parser just to give a helpful error
> message to people used to Python 2's print statement:
>
> >>> print 1, 2, 3
>   File "", line 1
> print 1, 2, 3
>   ^
> SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1,
> 2, 3)?
> >>>
>
> And IIRC there have been a number of aborted attempts at syntactic hacks
> to allow people to call functions (like print) without parentheses,
> although (I think) none of them made it into a PEP. The PEG parser makes
> this much simpler, because it can simply backtrack -- by placing the
> grammar rule for this syntax (tentatively called "call statement") last in
> the list of alternatives for "small statement" we ensure that everything
> that's a valid expression statement (including print() calls) is still an
> expression statement with exactly the same meaning, while still allowing
> parameter-less function calls, without lexical hacks. (There is no code in
> my prototype that checks for a space after 'print' -- it just checks that
> there's a name, number or string following a name, which is never legal
> syntax.)
>
> One possible extension I didn't pursue (yet -- dare me!) is to allow
> parameter-less calls inside other expressions. For example, my prototype
> does not support things like this:
>
> >>> a = (len "abc")
>   File "", line 1
> a = (len "abc")
>  ^
> SyntaxError: invalid syntax
> >>>
>
> I think that strikes a reasonable balance between usability and reduced
> detection of common errors.
>
> I could also dial it back a bit, e.g. maybe it's too much to allow
> 'C().foo x' and we should only allow dotted names (sufficient to access
> functions in imported modules and method calls on variables). Or maybe we
> should only allow simple names (allowing 'len x' but disallowing
> 'sys.getrefcount x'. Or maybe we should really only bring back print
> statements.
>
> I believe there are some other languages that support a similar grammar
> (Ruby? R? Raku?) but I haven't investigated.
>
> Thoughts?
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: 

[Python-ideas] Re: Bringing the print statement back

[Jonathan Crall]

> What's so special about print? It's just a function.

I'd argue it's a pretty special function given its history. Just because
it's used less frequently that something else doesn't mean it's not
"special" in some sense. `iter x` never worked, whereas `print x` used to
work, which is the only reason I'm giving it special status.

Regardless, I'd rather not have this feature. As you said it's not 100%
backwards compatible, so the usefulness is limited.

On Thu, Jun 11, 2020 at 11:49 AM Steven D'Aprano 
wrote:

> On Wed, Jun 10, 2020 at 08:00:26PM -0400, Jonathan Crall wrote:
> > I wouldn't mind if this *only *worked for the specific characters
> "print".
>
> I would. What's so special about print? It's just a function.
>
> I use `iter` much more than print. Should we make a special exception
> for only 'iter' too? Or instead?
>
> `print` is especially problematic, because zero-argument form of print
> is possible. This makes it a landmine waiting for the unwary:
>
> print x, y, z  # works
> print x  # works
> # now print a blank line
> print  # silent failure
>
>
> That's especially going to burn people who remember Python 2, where it
> did print a blank line instead of evaluating to the `print` object.
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ICVRZKRAPKRCNY7BTMQY7GRYV37ADQRA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4QNJB2JDVS7F2MMPYYV4HJXMG62GTIEJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditional with statements

[Jonathan Crall]

Thanks for the link, the rationale for rejection seems reasonable.

On Fri, Dec 11, 2020 at 8:19 PM Chris Angelico  wrote:

> On Sat, Dec 12, 2020 at 11:42 AM Jonathan Crall 
> wrote:
> >
> > I'm not sure if this has been asked / suggested before.
> >
> > I'm wondering if there is any interest in conditional or loop-based
> `with` statements. I think it could be done without a syntax change.
> >
> > ### Napkin proposal
> >
> > Context managers could define `__if__` or `__for__`, and if those dunder
> methods were defined, then the `with` statement would either behave like a
> conditional or a loop.
> >
> > If `__if__` was defined then
> >
> > ```
> > with Ctx():
> > print('hi')
> > ```
> >
> > would only print `hi` if `__if__` returned True. This doesn't require a
> syntax change.
>
> This part has been proposed before:
> https://www.python.org/dev/peps/pep-0377/
>
>
> > The `__for__` variant would likely need a minor syntax change.
> >
> > ```
> > with item in Ctx():
> > print(item)
> > ```
> >
> > The `__for__` method is a generator that generates arguments of a loop.
> The item will be printed as many times as there are items generated by
> `__for__`.
> >
>
> Not sure that this one has, but it's basically just a context manager
> and a for loop, so I'm not really sure how much you'd gain over just
> using the two constructs independently, given that there'd then be
> massive confusion over "when should I use 'with item in thing' and
> when should I use 'for item in thing'?".
>
> For many use cases, it may be best to write the body as a function,
> which can then be called more than once. You can decorate a function
> in order to do whatever you like, and the return value from the
> decorator could be whatever stats you want to provide (there's no rule
> says that a function decorator has to return a function!).
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/Q7WZ7FY3ZSINGMHPI6ULQA6L5MJZ6HL3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3SGCC2KR6CL2IUON7BENQBTNK5MWDJYO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Conditional with statements

[Jonathan Crall]

I'm not sure if this has been asked / suggested before.

I'm wondering if there is any interest in conditional or loop-based `with`
statements. I think it could be done without a syntax change.

### Napkin proposal

Context managers could define `__if__` or `__for__`, and if those dunder
methods were defined, then the `with` statement would either behave like a
conditional or a loop.

If `__if__` was defined then

```
with Ctx():
print('hi')
```

would only print `hi` if `__if__` returned True. This doesn't require a
syntax change.

The `__for__` variant would likely need a minor syntax change.

```
with item in Ctx():
print(item)
```

The `__for__` method is a generator that generates arguments of a loop. The
item will be printed as many times as there are items generated by
`__for__`.


### Use Cases
This would simplify usage of my Timerit and ubelt library.

The timerit module defines ``timerit.Timerit``, which is an object that is
iterable. It has an ``__iter__`` method that generates
``timerit.TimerTimer``
objects, which are context managers.

>>> import math
>>> from timerit import Timerit
>>> for timer in Timerit(num=200, verbose=2):
>>> with timer:
>>> math.factorial(1)

The timer context manager measures how much time the body of it takes by
"tic"-ing ``__enter__`` and "toc"-ing on ``__exit__``. The underlying object
has access to the context manager, so it is able to read its measurement.
These
measurements are stored and then we compute some statistics on them. Notably
the minimum, mean, and standard-deviation of grouped (batched) running
times.

Unfortunately the syntax is one line and one indent bulker than I would
prefer.
However, a more consice version of the synax is available.

>>> import math
>>> from timerit import Timerit
>>> for _ in Timerit(num=200, verbose=2):
>>> math.factorial(1)

In this case the measurement is made in the `__iter__` method ``Timerit``
object itself, which I believe contains slightly more overhead than the
with-statement version. (I should test to determine if this is the case).

In the case where it does make a difference, a cool syntax might look like:
>>> import math
>>> from timerit import Timerit
>>> with timer in Timerit(num=200, verbose=2):
>>> math.factorial(1)

The other case is that my ``ubelt.Cacher`` library. Currently it requires 4
lines of boilerplate syntax.

>>> import ubelt as ub
>>> # Defines a cache name and dependencies, note the use of
`ub.hash_data`.
>>> cacher = ub.Cacher('name', cfgstr=ub.hash_data('dependencies'))
# boilerplate:1
>>> # Calling tryload will return your data on a hit and None on a miss
>>> data = cacher.tryload()
# boilerplate:2
>>> # Check if you need to recompute your data
>>> if data is None:
 # boilerplate:3
>>> # Your code to recompute data goes here (this is not
boilerplate).
>>> data = 'mydata'
>>> # Cache the computation result (pickle is used by default)
>>> cacher.save(data)
# boilerplate:4

But a conditional ``with`` syntax would reduce boilerplate to 3 lines.

>>> import ubelt as ub
>>> with ub.Cacher('name', cfgstr=ub.hash_data('dependencies')) as
cacher:
>>> data = 'mydata'
>>> cacher.save(data)
>>> data = cacher.data

I'm sure there are a lot of viable syntax variations, but does the idea of
a conditional or loop aware "with" statement seem like a reasonable
language proposal?

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OMFS77O4KRCJZBPMSEAMEKIQ67GTD5PH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditional with statements

[Jonathan Crall]

@Iasizoillo

> Only if you don't setup vars like official example* or not calc some
statistic after timered block

I wrote the official docs, so I'm aware of that case. In the case where
setup
variables are required it make sense to me to have an extra level of
indentation. It just bothers me that I still need extra indentation when I
*don't* need setup variables.

> In your example this is true by luck,

Yes, but I also chose a verbose and explicit invocation of `Timerit` because
people may not be familiar with what the default arguments are. I'm also not
opposed to line wrapping e.g.

import timerit
class MyClass:
def myfunc():
# Next line is 79 chars long, at limit of restricted pep8
for timer in timerit.Timerit(
  num=100, bestof=3, verbose=1): with timer:
[code]

Although, at this point I would have refactored the code to look like this:

import timerit
class MyClass:
def myfunc():
ti = timerit.Timerit(num=100, bestof=3, verbose=1)
for timer in ti: with timer:
[code]


Furthermore, having one line that is over the character limit (whether it be
79, 99, or whatever), is a lot easier to deal with than multiple lines in
the
[code] block.


@Greg Ewing

> I don't know why some people seem to be so afraid of indentation levels

There are a few reasons for me.  I don't like when my linter yells at me.
But
more importantly: I don't like when lines wrap around (or I have to
horizontally scroll) when I have two files open on a portrait monitor in
vertical split mode and lines wrap around. On my 4K setup, I have 97
characters
of horizontal space to play with in the best case (it's worse if I have a
NerdTree sidebar open). On my older monitor this situation is worse.


> Remember, you don't *have* to indent by 4 spaces. I often write Python
with
> 2-space indents, which is quite readable and gives me twice as many
> indentation levels to play with.


This is a good point, and one I have considered. I know I *could*, and I
don't
have a great argument against this point. It feels wrong though. Perhaps in
the
same way that `if [cond]: with [obj]:` feels wrong to others? Or maybe its
the
more grounded facts that (1) I prefer a consistent indentation level and
(2) I
set my editor to use a 4 space indentation level and mixing and matching 2
or 4
space indentations mess with that. I do think this is one of the best
arguments
against my idea, but at the same time I feel like the idea has enough merit
where this argument against it doesn't kill it.



@Stephen J Turnbull
> A big issue for me is that "with" doesn't commute with "if".


That would be a huge problem if "if" commuted with "with"!

The order of nesting *should* be the order of appearance. Your example:

for line in f with open('file') as f:
process(line)

Should be rewritten as:

with open('file') as f: for line in f:
process(line)

The other way around, the with would happen in every iteration of the loop.


> Notice that once you combine them into a single statement, you can choose
either semantics, which is a bug magnet

To be fair, coding is a bug magnet. And yes, there would need to be a rule
that
defines semantics. Again, I would propose that "the order of appearance is
the
order of nesting" should be that rule.

> IMO this syntax probably should be limited to the particular case where
there's a context manager

Having considered this more, I prefer the style where the grammar
effectively
allows chaining of nesting statements. It seems a lot more elegant and
consistent than constraining this to just conditionals. If implemented, I
personally would likely only use this for the context manager case, but I
know
people like Jeremy Howard would have a field day with it.


@David Mertz

> Code is READ far more often than it is written!

Yes! I find myself reading my double indented context blocks /
non-contiguous
caching logic a lot and wishing they were smaller / contiguous.

> I don't like the actual proposal. It weirdly combines almost orthogonal
> concepts into the same block statement.

This is a good point. I think the syntax -- as with any other programming
construct -- should be used judiciously. I envision it used in cases like
the
ones I mentioned where the `if` and `with` deal with correlated information.
`if stamp.expired():` and `with stamp:` are not orthogonal.



@Christopher Barker

> Are you suggesting that we add syntax to Python that makes it easier to
keep the number of levels of indentation down?

Yes, I think the core essence of this proposal is a way to keep indentation
down. Especially in cases where there are multiple levels of (short) nesting
statements with no other logic in between.  This is a fairly common case,
and
having the option to reduce required indentation seems like it might be
nice.
Obviously this isn't critical for the Python grammar, but at this point what
is?  It's such a 

[Python-ideas] Re: Add an export keyword to better manage __all__

[Jonathan Crall]

I'd like to give a shoutout to a package I wrote called mkinit, which helps
autogenerate explicit `__init__.py` files to mitigate some of the issues
you mentioned (especially with respect to `from blah import *`).

https://github.com/Erotemic/mkinit

On Sat, Apr 10, 2021 at 3:08 PM Guido van Rossum  wrote:

> On Sun, Mar 14, 2021 at 10:55 PM Ethan Furman  wrote:
>
>> I completely agree with this.  One of the hallmarks of Python is the
>> ability to query, introspect, and modify Python code.  It helps with
>> debugging, with experimenting, with fixing.  Indeed, one of the few
>> frustrating bits about Python is the inability to work with the C portions
>> as easily as the Python portions (note: I am /not/ suggesting we do away
>> with the C portions).
>>
>> What would be the benefits of locking down modules in this way?
>
>
> I owe you an answer here. For large projects with long lifetimes that
> expect their API to evolve, experience has taught  that any part of the API
> that *can* be used *will* be used, even if it was clearly not intended to
> be used. And once enough users have code that depends on reaching into some
> private part of a package and using a method or attribute that was
> available even though it was undocumented or even clearly marked as
> "internal" or "private", if an evolution of the package wants to remove
> that method or attribute (or rename it or change its type, signature or
> semantics/meaning), those users will complain that the package "broke their
> code" by making a "backwards incompatible change." For a casually
> maintained package that may not be a big deal, but for a package with
> serious maintainers this can prevent ordered evolution of the API,
> especially since a package's developers may not always be aware of how
> their internal attributes/methods are being used or perceived by users. So
> experienced package developers who are planning for the long term are
> always looking for ways to prevent such situations (because it is
> frustrating for both users and maintainers). Being able to lock down the
> exported symbols is just one avenue to prevent disappointment in the future.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/XDLAAQKWDSPOJ6HKVTBEZEK6KYM5PQGP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TNTM3XCN6VBL3WT7ZT5VFG2QJ6FNJ2LM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditional with statements

[Jonathan Crall]

I ran into another case where I wish I had some sort of conditional if.

I was writing code to cache autogenerated demodata.

import ubelt as ub
from os.path import join
import json
kwargs = {'example': 'config'}

dpath = ub.ensure_app_cache_dir('my_modname', 'demodata')
fpath = join(dpath, 'data.json')

stamp = ub.CacheStamp('demodata', depends=kwargs, dpath=dpath)
if stamp.expired():
data = {
'complicated': 'data',
'key1': 'val1',
'key2': 'val2',
'keyN': 'valN',
}
with open(fpath, 'w') as file:
json.dump(data, file)
stamp.renew()
else:
with open(fpath, 'r') as file:
data = json.load(file)

I really wish I didn't have to have that stamp.renew() at the end of the if
block. It unnecessary boilerplate --- the caching logic would all be in a
contiguous block if not for the need for this. It wastes a little bit of
vertical, which while not critical, is a consideration.

I want to focus on the context:

stamp = ub.CacheStamp('demodata', depends=kwargs, dpath=dpath)
if stamp.expired():
[code]
stamp.renew()


I could make `CacheStamp` a context manager and tell it to do it in its
`__exit__` clause, as such:

stamp = ub.CacheStamp('demodata', depends=kwargs, dpath=dpath)
if stamp.expired():
with stamp:
[code]

This removes the need for the `stamp.renew`, makes the logic contiguous,
has the same vertical space, however, it adds a TON of horizontal space
depending on the complexity of the logic. It's often beneficial to minimize
nesting and try restricting it 2 or 3 levels.

BUT if we had just a tiny bit of new syntax rules we could write something
like this:

stamp = ub.CacheStamp('demodata', depends=kwargs, dpath=dpath)
if stamp.expired() with stamp:
[code]

There are even more conservative changes, like requiring a colon before
the `with`: `if stamp.expired(): with stamp:`

I know this is just saving a line. But I use this pattern with
`ubelt.CacheStamp` and `ubelt.Cacher` frequently, and I always feel a
strong want for this syntactic sugar as I'm writing it. Wasting that
horizontal space is not an option, and I really would like the caching
logic to be contiguous.

To be clear, in the proposed syntax:

if [condition] with [obj]:
[code]

Would behave exactly as:

if [condition]:
with [obj]:
[code]

Is there any chance that this conditional context manager syntax might be
considered? Does anyone but myself think this might be a good idea?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TR2JBOMPXBCAZOKJUB67OIU27Z6FTK6G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Conditional with statements

[Jonathan Crall]

@MRAB

> It's not that much shorter. You wouldn't be saving much typing or space.

IMO, that's a weak point. The amount of indentation you need doubles, and
that is significant.
Also, common style guides and best practices recommend no more than 80
chars per line.
Losing 5.5% of my available space when I'm already in a method per line is
not nothing.

With modern editors there is no typing cost to either method. I never
claimed this would save typing. But it does clearly save a lot of
horizontal space (and 1 vertical line, which is not nothing).


@Paul Sokolovsky

> but it's proverbial Freudian slip ...

Yes, I made a typo. Try not to judge too harshly because of a dyslexic
failure to catch a grammatical issue.

> it seems that some people just can't get at peace with Python's
indentation-based syntax,

And I'm all on board with indentation based syntax. I'm not typically one
to try and make Python work like other languages. My motivation simply
stems from the fact that I've been bothered how a with directly after an if
costs me 8 spaces instead of 4. I strongly feel like this would be a good
change, and I am going to fight at least a little bit to defend the idea,
or at least try to advocate for it and help refine the syntax into
something better. I do think this new syntax is a lot better than my
original proposal, which is admittedly not a good idea, It's possible this
could be tweaked more to improve it.

Indentation makes beautiful code, but it's not free. The more horizontal
space you take --- the more nesting in your code ---
the higher its complexity. I don't feel that having a with statement
immediately after an if (or a for loop for that matter) is adding
much complexity, but if forces double indentation of whatever code was
going to be after the for loop.


@Iasizoillo

> Other thing strange about proposal is: why only with after if?

However isn't the fact that you *can* do

if 1: return 1

also weird? I think something like `if [condition] for x in [iterable]`
would be reasonable. I also suppose `if [condition] while [other-condtion]`
have some use as well. I can also see uses for something like

`for var in [iterable] with var:` being useful.

I can actually think of a specific case where I would use it.

Currently my timerit package requires syntax like:

import timerit
for timer in timerit.Timerit(num=100, bestof=3, verbose=1):
with timer:
[code]

But that could be simplified to


import timerit
for timer in timerit.Timerit(num=100, bestof=3, verbose=1) with timer:
[code]


And free up one level of indentation (very useful if you are trying to keep
under 80 chars per line)

Perhaps the new syntax rule is as simple as, if a keyword that would
normally require a new scope is placed where the colon would be for another
statement that defines a new scope, then any code indented afterwords is
treated as in the scope of all of the statements. That means you could do
something crazy like:

if foo if bar or baz with biz if buz:
[code]

In that example the biz context manager would only trigger if `foo and (bar
or baz)` and if it did and buz was true then [code] would execute.
Obviously I wouldn't recommend doing something like this stylistically, it
would be a "bad code" usage of the proposed syntax (but all syntax has ways
that it can be used to write bad code). This example does make me lean
slightly more in favor of requiring the colon before the next "scoping"
keyword is used, e.g.:


if foo: if bar or baz: with biz: if buz:
[code]

Note that by convention an `else` would need to correspond to the outermost
(or alternatively innermost) scoping keyword, so for

if foo: if bar or baz: with biz: if buz:
[code1]
else:
[code2]

[code2] only executes if `not foo`.

On Sun, Feb 7, 2021 at 1:28 AM Paul Sokolovsky  wrote:

> Hello,
>
> On Sat, 6 Feb 2021 19:57:33 -0500
> Jonathan Crall  wrote:
>
> > I ran into another case where I wish I had some sort of conditional
> > if.
>
> Right, conditional "if" is exactly what we miss in Python.
>
> Yes, it's a typo, but it's proverbial Freudian slip, shows what stays
> behind such proposals - un[spell]checked desire to add some tautology
> to the language.
>
> > Wasting that horizontal space is not an option
>
> Reading things like this, it seems that some people just can't get at
> peace with Python's indentation-based syntax, and would jump in no time
> to an alternative syntax which allows to smack indentation.
>
> It was also said many times, that it you want some trivial variety on
> the existing language, pick up your favorite macro package and go
> ahead, coding trivial macros in trivial.
>
>
> --
> Best regards,
>  Paul  mailto:pmis...@gmail.com
>


-- 
-Dr. Jon Crall (him)
___

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

That's effectively every module I write. I don't think that's uncommon or
bad practice.

On Sat, Oct 2, 2021, 12:05 AM Chris Angelico  wrote:

> On Sat, Oct 2, 2021 at 2:01 PM Jonathan Crall  wrote:
> >
> > @Steven D'Aprano Yes my original post is somewhat pithy and has some
> harsh words for decorators. Like many I have some opinions about code.
> Don't take them too seriously if they disagree with your experience, but I
> do hold my opinions for reasons, and I will explain some of them, but I
> don't want to derail my first python ideas proposal that has gotten some
> semblance of a positive reception.
> >
> > So I will quickly explain my terse statements in a more professional
> manner. And I will note again, these are opinions.
> >
> > * Avoid global variables, they make state difficult to reason about.
> Plenty of opinions have been written about this.
> >
> > * Code runs faster when it's in a function then when it is in a global
> scope. I don't have the reference on hand, but I can get it if anyone's
> interested. Python is a scripting language, but it's used heavily in
> production, It's nice if the efficient way of doing something is natural,
> intuitive, and easy.
> >
> > It's true that it's just a two-line pattern, and not all simplifications
> of two line patterns make it into the python standard. But this is an
> extremely common two line pattern. One that's often written less
> efficiently than it could be. If you already wrote the if statement for
> name, a new programmer is not incentivized to write a separate main
> function and then call it, it would be simpler if there was some construct
> that just declared this is the main function, and then the code inside the
> function is executed.
> >
>
> But it's ONLY necessary when a single Python file is used as both a
> script and a module. Bear that in mind.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/B3RDNS6HB5UIPIGE5E2DEWL5JPC63W24/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ALYRAXOUTSCCOTELRQUZNIICLM3NPAEL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

I was curious if / what sort of proposals have been considered for
simplifying the pattern:

```
def main():
...

if  __name__ == "__main__":
main()
```

I imagine this topic must have come up before, so I'd be interested in any
relevant history.

But unless I'm missing something, it seems like adding some easier
alternative to this cumbersome entrypoint syntax would be worth considering.

My motivation for writing this suggestion is in an attempt to stop a common
anti-pattern, where instead of defining a `main` function (or a function by
any other name) an simply calling that by adding the above two lines, a lot
of Python users I work with will just start dumping their logic into the
global scope of the module.

Needless to say, this can have consequences. If there was some default
builtin, let's call it `__main__` for now (open to suggestions), that took
a function as an argument and conditionally executed it if `__name__ ==
"__main__"` in the caller's scope, that would allow us to simplify the
above boilerplate to a single line with no extra indentation:

```
def main():
...

__main__(main)
```

In addition to being simpler, it would allow users to avoid the trap of
adding logic that impacts the global scope. It would also save me some
keystrokes, which I'm always grateful for.

Furthermore, it could be used as a decorator (and the use-case wouldn't be
unreasonable!), and we all know how much new Python users love decorators
when they find out about them.

```
@__main__
def main():
...
```

Maybe having such a builtin would discourage globals and help new users get
the use-decorators-everywhere bug out of their system.

--
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FKQS2NEI5RQMTX53N77KQQDFZ6HZONXU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

@Steven D'Aprano  Yes my original post is somewhat
pithy and has some harsh words for decorators. Like many I have some
opinions about code. Don't take them too seriously if they disagree with
your experience, but I do hold my opinions for reasons, and I will explain
some of them, but I don't want to derail my first python ideas proposal
that has gotten some semblance of a positive reception.

So I will quickly explain my terse statements in a more professional
manner. And I will note again, these are opinions.

* Avoid global variables, they make state difficult to reason about. Plenty
of opinions have been written about this.

* Code runs faster when it's in a function then when it is in a global
scope. I don't have the reference on hand, but I can get it if anyone's
interested. Python is a scripting language, but it's used heavily in
production, It's nice if the efficient way of doing something is natural,
intuitive, and easy.

It's true that it's just a two-line pattern, and not all simplifications of
two line patterns make it into the python standard. But this is an
extremely common two line pattern. One that's often written less
efficiently than it could be. If you already wrote the if statement for
name, a new programmer is not incentivized to write a separate main
function and then call it, it would be simpler if there was some construct
that just declared this is the main function, and then the code inside the
function is executed.

For giant single file scripts, that are just protected by an if name equals
main, this could be a big performance improvement. Although I imagine in
the average case it won't impact that much. But there may be some small
benefit. The biggest benefit will be in the clarity of the code, the state
will be easier to reason about because there will be less globals.

My initial proposal could absolutely be implemented in pure Python, but I
do think I like the idea of just having a special dunder main function.

I am an experienced user who avoids decorators. They make it difficult to
statically parse the code. They are useful, but when I was younger I did
fall in the trap of abusing them. And I do think there is something to be
said for statically parsable code. Having to guarantee that when you see a
function definition, that that function definition won't change or be
dynamically rewritten, can be nice. There's value in dynamic code,
decorators are a good concept, and Python is better for them, but I do find
it fun to take jabs at them from time to time.

So yes I was being a bit sarcastic and part of the proposal, but I was
serious when I proposed using that form of dunder main as a decorator.
However now I do favor Python just checking that there is a dunder main
function, and calling it. That would be in the spirit of dunder getattr,
which imo is one of the best improvements Python has made.



On Fri, Oct 1, 2021, 9:01 PM Steven D'Aprano  wrote:

> On Fri, Oct 01, 2021 at 03:35:26PM -0400, Jonathan Crall wrote:
>
> > My motivation for writing this suggestion is in an attempt to stop a
> common
> > anti-pattern, where instead of defining a `main` function (or a function
> by
> > any other name) an simply calling that by adding the above two lines, a
> lot
> > of Python users I work with will just start dumping their logic into the
> > global scope of the module.
> >
> > Needless to say, this can have consequences.
>
> What sort of consequences are you thinking of, under what circumstances?
>
> Remember, Python is first and foremost a scripting language. There's
> nothing wrong with writing scripts that run in the global scope.
>
>
> > If there was some default
> > builtin, let's call it `__main__` for now (open to suggestions), that
> took
> > a function as an argument and conditionally executed it if `__name__ ==
> > "__main__"` in the caller's scope, that would allow us to simplify the
> > above boilerplate to a single line with no extra indentation:
> >
> > ```
> > def main():
> > ...
> >
> > __main__(main)
> > ```
>
> You don't need `__main__` to be built into the interpreter. It's a
> two-line pattern: not every two line function needs to be a builtin.
>
>
> ```
> def __main__(func):
> if __name__ == '__main__': func()
> ```
>
>
> Put that in a module, then:
>
> ```
> from module import __main__
> ```
>
> in all your scripts.
>
> If you're worried about keystrokes, and your editor doesn't have a
> "snippets" functionality, you can put your `__main__` in your
> PYTHONSTARTUP file or sitecustomize.py and have it loaded automatically
> into builtins.
>
> https://pymotw.com/3/site/
>
> ```
> import builtins
> builtins.__main__ = __main__
> ```
>
>
> > In addition to being simpler, it wo

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

Yes, it seems like having special handling of `def __main__` is an
extension in the spirit of `package/__main__.py `.

On Fri, Oct 1, 2021 at 3:43 PM Thomas Grainger  wrote:

> FYI you can already use package/__main__.py which is runnable with `python
> -m package` and you don't need the `if __name__ == "__main__":`
> https://docs.python.org/3.10/library/__main__.html#main-py-in-python-packages
>
> On Fri, 1 Oct 2021, 20:39 Paul Bryan,  wrote:
>
>> How about the following?
>>
>> def __main__():
>>
>> ...
>>
>>
>> Behavior:
>>
>> 1. Load module as normal.
>> 2. If __name__ is "__main__" or module is named in python -m, call
>> __main__ function.
>>
>> Paul
>>
>>
>> On Fri, 2021-10-01 at 15:35 -0400, Jonathan Crall wrote:
>>
>> I was curious if / what sort of proposals have been considered for
>> simplifying the pattern:
>>
>> ```
>> def main():
>> ...
>>
>> if  __name__ == "__main__":
>> main()
>> ```
>>
>> I imagine this topic must have come up before, so I'd be interested in
>> any relevant history.
>>
>> But unless I'm missing something, it seems like adding some easier
>> alternative to this cumbersome entrypoint syntax would be worth considering.
>>
>> My motivation for writing this suggestion is in an attempt to stop a
>> common anti-pattern, where instead of defining a `main` function (or a
>> function by any other name) an simply calling that by adding the above two
>> lines, a lot of Python users I work with will just start dumping their
>> logic into the global scope of the module.
>>
>> Needless to say, this can have consequences. If there was some default
>> builtin, let's call it `__main__` for now (open to suggestions), that took
>> a function as an argument and conditionally executed it if `__name__ ==
>> "__main__"` in the caller's scope, that would allow us to simplify the
>> above boilerplate to a single line with no extra indentation:
>>
>> ```
>> def main():
>> ...
>>
>> __main__(main)
>> ```
>>
>> In addition to being simpler, it would allow users to avoid the trap of
>> adding logic that impacts the global scope. It would also save me some
>> keystrokes, which I'm always grateful for.
>>
>> Furthermore, it could be used as a decorator (and the use-case wouldn't
>> be unreasonable!), and we all know how much new Python users love
>> decorators when they find out about them.
>>
>> ```
>> @__main__
>> def main():
>> ...
>> ```
>>
>> Maybe having such a builtin would discourage globals and help new users
>> get the use-decorators-everywhere bug out of their system.
>>
>> --
>> -Dr. Jon Crall (him)
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/FKQS2NEI5RQMTX53N77KQQDFZ6HZONXU/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/RNB5YYTGQIV4CRPUZEZTADKG7WJ4YY3B/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WQJRWP3JT5HNRXO6BGG45MVRTDBQRJUB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

That would also make a lot of sense. That way has fewer characters without
sacrificing clarity, which is a +1 in my book.

On Fri, Oct 1, 2021 at 3:38 PM Paul Bryan  wrote:

> How about the following?
>
> def __main__():
>
> ...
>
>
> Behavior:
>
> 1. Load module as normal.
> 2. If __name__ is "__main__" or module is named in python -m, call
> __main__ function.
>
> Paul
>
>
> On Fri, 2021-10-01 at 15:35 -0400, Jonathan Crall wrote:
>
> I was curious if / what sort of proposals have been considered for
> simplifying the pattern:
>
> ```
> def main():
> ...
>
> if  __name__ == "__main__":
> main()
> ```
>
> I imagine this topic must have come up before, so I'd be interested in any
> relevant history.
>
> But unless I'm missing something, it seems like adding some easier
> alternative to this cumbersome entrypoint syntax would be worth considering.
>
> My motivation for writing this suggestion is in an attempt to stop a
> common anti-pattern, where instead of defining a `main` function (or a
> function by any other name) an simply calling that by adding the above two
> lines, a lot of Python users I work with will just start dumping their
> logic into the global scope of the module.
>
> Needless to say, this can have consequences. If there was some default
> builtin, let's call it `__main__` for now (open to suggestions), that took
> a function as an argument and conditionally executed it if `__name__ ==
> "__main__"` in the caller's scope, that would allow us to simplify the
> above boilerplate to a single line with no extra indentation:
>
> ```
> def main():
> ...
>
> __main__(main)
> ```
>
> In addition to being simpler, it would allow users to avoid the trap of
> adding logic that impacts the global scope. It would also save me some
> keystrokes, which I'm always grateful for.
>
> Furthermore, it could be used as a decorator (and the use-case wouldn't be
> unreasonable!), and we all know how much new Python users love decorators
> when they find out about them.
>
> ```
> @__main__
> def main():
> ...
> ```
>
> Maybe having such a builtin would discourage globals and help new users
> get the use-decorators-everywhere bug out of their system.
>
> --
> -Dr. Jon Crall (him)
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/FKQS2NEI5RQMTX53N77KQQDFZ6HZONXU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PZCGQK5BM7J6EULVDUSZNFWVPTQ47MZL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Jonathan Crall]

@Marc-Andre

One of the motivations of this proposal is to incentivize writing code in a
local scope, rather than a global one. People in this thread have made the
argument "Python is a scripting language, so it should do that". Ok, fine.
Yes `print('hello world')` is more idiomatic than the one inside of a
function, but any real work done in Python quickly discards that global
simplicity.

Variable assignment in a global scope is slower than the equivalent code in
a local scope, due to the STORE_FAST bytecode being used inside a function
and STORE_NAME being used in the global scope:
https://stackoverflow.com/questions/11241523/why-does-python-code-run-faster-in-a-function

And it is true that this isn't a huge performance hit, and it is entirely
avoidable by simply adding the

```
def main():
...

if __name__ == '__main__':
main()
```

boilerplate yourself, but it feels inelegant to me. I suppose it is a
subtle reminder that Python is a scripting language, and we are coercing it
into a library. So I can understand that argument.


One of the motivations for me posting this was a several students I was
working with were putting all of their logic in the `if __name__ ==
'__main__'` block, defining global variables in them, and then relying on
those global variable existing in other places in the code. This did what
they expected when they were running their code as a script, but when I
tried to import it as a module, I immediately encountered errors. My
thought is that these errors would have been more obvious to the students
if they were writing their logic (context is training a neural network) in
a main function, rather than in the global scope.


@Christopher Barker

> The if __name__ block is only required for a Python file to be both a
module and a script.
> That’s actually a pretty uncommon thing— if it’s a module to be imported
by other modules, then

I suppose I do an uncommon thing fairly often. I work with neural networks
and I generally construct a package with a pkg/fit.py pkg/predit.py and
pkg/evaluate.py which all operate as both a module and a script. I'll run
`python -m pkg.fit [args]` to train a network but I'll also `from pkg
import fit`, especially when testing. I think if you are testing an
executable module you almost always want to have part of it be importable
so it can be unit tested. And I think the natural argument against that
here, is, well why don't you put that in a `__main__.py`? To which my
response is that `pkg` has multiple entry points, so that would require
some sort of modal boilerplate, which is useful in many cases and I have
done it, but it works against the simplicity of the case where it makes
intuitive sense that the package module should be accessible as a module
and a library.

Retrospectively, I would agree that this is probably uncommon, but if
nothing else comes out of this discussion, I would want to encourage Python
users to write Python files as both modules and executables more often.
Having lots of entry points can be handy, and makes a lot of sense in many
cases (e.g. whenever you have a main module with more than one function, it
almost always makes sense to be able to import it to unit test the
components). I would also encourage reserving the "__main__.py" file to be
"the one true" entrypoint, or at least the primary public facing
entrypoint. I find this pattern strikes a nice balance between having lots
of entry points (which is good for people familiar with the code base) and
having one obvious entrypoint for people learning the codebase.

On Sun, Oct 3, 2021 at 10:20 AM David Mertz, Ph.D. 
wrote:

> On Sun, Oct 3, 2021, 1:46 AM Christopher Barker
>
>> The if __name__ block is only required for a Python file to be both a
>> module and a script.
>> That’s actually a pretty uncommon thing— if it’s a module to be imported
>> by other modules, then it probably should be part of a package, and if the
>> functionality needs to be made available as a script, there can be another
>> module just for that.
>>
>
> I think all the slightly different boilerplate forms suggested are
> needless. So on the main topic, -1.
>
> But as a note, I usually put a __name__=='__main__' block at the foot of
> my library code.
>
> The few functions in this particular file presumably do something useful,
> and hopefully composable. Why not allow command-line users to do just that
> one thing by itself? Sure, the predominant use is within a library, but
> being able to try out this functionality usually helps my development, and
> is sometimes useful to have around and composable in bash.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/6XGPJH57YY4N2AC6P4VY67VNYPINZKQD/
> Code of Conduct: 

[Python-ideas] Re: Adding pep-8-casing-compliant aliases for unittest and logging

[Jonathan Crall]

IMO (a subset of) pytest is the superior unit-testing framework. Just
having functions prefixed with `test_` is easy and intuitive. It also works
well with xdoctest, but I would use it even if that wasn't the case. (That
being said, I think the current implementation of fixtures - where you
can't easily make an instance of the object outside of the pytest framework
- are an anti-pattern, and I will die on that hill).

As for logging, I would love a redesigned logging module. The main issues I
have with the current one are: (1) I don't like being forced to put logging
initialization at every entry point in my program/library, I wish there was
a better way to do that, even though I can't think of what that is. (2)
When I do enable logging I often get junk from other libraries using it
that I don't need. (3) I want first-class support for instance-level
logging objects that belong to a specific object instance, this can sort-of
be done with the current interface, but it's a bit clunky.

But as to the original intent of this post, I agree with Stephen Turnbull
that foolish pep8 consistency is not a big enough reason to touch the
stdlib. And honestly, a better logger would likely be best handled by a
third party library, at least for the initial phases of its development and
field testing. If it got wide acceptance and was simple enough, perhaps
making its way into the stdlib could be considered. But I don't think a
good alternative even exists right now.

There is something to be said for replacing unittest with a better
no-third-party solution, and I think pytest is a bit too big to bring into
the stdlib. But perhaps a subset of it (pytest lite?) might be isolated and
considered for inclusion down the line?


On Fri, Nov 12, 2021 at 7:49 AM Stephen J. Turnbull <
stephenjturnb...@gmail.com> wrote:

> Christopher Barker writes:
>
> +1 to your earlier comment along the lines of "Java flavor is an issue
> (sometimes) worth fixing, non-PEP8 identifiers is not."  (I do not
> claim Chris signs on to my formulation, I'm riffing on his. ;-)
>
>  > Note that a LOT of major projects dumped unittest years ago— first
>  > for nose and now usually pytest. No one uses a third party library
>  > for no reason :-)
>
> I can't speak to PyTest, but Mailman uses tox and nose2 -- a lot -- as
> well as doctests, and it's good enough that so far nobody has
> suggested moving to something else.
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/FJCXM4UQOMB6JAVFN3HCFWTSQXMJKRKF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ENOBGONUYEK6VC2MVMIMGQFFTMWADGCW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Revisiting a frozenset display literal

[Jonathan Crall]

Not a huge fan of an f-prefix for a frozen set (I prefer just recognizing
the case and optimizing the byte code, I don't think frozensets are used
often enough to justify its own syntax), but I love {,} for an empty set.

On Tue, Jan 18, 2022 at 4:13 PM Rob Cliffe via Python-ideas <
python-ideas@python.org> wrote:

>
>
> On 18/01/2022 19:42, MRAB wrote:
> > On 2022-01-18 18:54, Neil Girdhar wrote:
> >> Even if f{1} creates a frozenset, I don't think f{} should create a
> >> frozenset.  I think it makes more sense to keep f{1: 2} open for
> >> frozendict if it ever makes it in.  Also, {} should be consisten with
> >> f{} (both should create dicts).  If you want an empty frozenset, you
> >> would have to do it the same way you do it for sets: either
> >> frozenset() or f{*()}.
> >>
> > [snip]
> > An alternative might be to allow {,} for an empty set, which would
> > then let you have f{,} for an empty frozenset.
> I like it!
> Rob Cliffe
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HXJXYO5A7CO6HZHBQEFYS5Q7IHKSY7Y3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JMTIQSP6IKXZWZSWWWJME2HLPASJVMXM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto dedent -c arguments.

[Jonathan Crall]

I can't think of a case where the check for "\n" would result in a false
negative. It likely makes sense to do that to keep python startup as
lightweight as possible.

I've made a PR that implements a proof-of-concept:
https://github.com/python/cpython/pull/103998

It's currently done via the Python C-API, but in terms of keeping things
lightweight, I might need some help on how to effectively use the wide
c-string API to implement this.


On Mon, Apr 10, 2023 at 4:46 AM Barry  wrote:

>
>
> On 10 Apr 2023, at 02:18, Jonathan Crall  wrote:
>
> 
> There's no question that there are lots of ways you can work around the
> issue (I think the `if 1:` method is the easiest and most portable, but it
> still is boilerplate which can be confusing if you don't know why it's
> there). The question is: would enough people benefit from this to add the
> feature to CPython?
>
>
> A PR that has an implementation might push this forward.
> I wonder if in the C code of -c treating a string that starts with \n
> could trigger the dedent?
>
> Barry
>
>
>
>
> On Sun, Apr 9, 2023 at 4:09 PM Barry Scott  wrote:
>
>>
>> On 04/04/2023 15:18, Jonathan Crall wrote:
>>
>> I have what I think is a fairly low impact quality of life improvement to
>> suggest for the python CLI.
>>
>> When I'm not working in Python I tend to be working in bash. But often I
>> want to break out and do something quick in Python. I find the `python -c `
>> CLI very useful for this. For one liners it's perfect. E.g.
>>
>> NEW_VAR=$(python -c "import pathlib;
>> print(pathlib.Path('$MYVAR').parent.parent)")
>>
>> And even if I want to do something multi-line it's pretty easy
>>
>> NEW_VAR=$(python -c "
>> import pathlib
>> for _ in range(10):
>> print('this is a demo, bear with me')
>> ")
>>
>> But the problem is when I'm writing bash inside a function or some other
>> nested code, I would like to have nice indentation in my bash file, but if
>> I write something like this:
>>
>> mybashfunc(){
>> python -c "
>> import pathlib
>> for _ in range(10):
>> print('this is a demo, bear with me')
>> "
>> }
>>
>> I get `IndentationError: unexpected indent`.
>>
>> This means I have to write the function ugly like this:
>>
>> mybashfunc(){
>> python -c "
>> import pathlib
>> for _ in range(10):
>> print('this is a demo, bear with me')
>> "
>> }
>>
>> Or use a helper function like this:
>>
>> codeblock()
>> {
>> __doc__='
>> copy-pastable implementation
>> Prevents indentation errors in bash
>> '
>> echo "$1" | python -c "import sys; from textwrap import dedent;
>> print(dedent(sys.stdin.read()).strip('\n'))"
>> }
>>
>> mybashfunc(){
>> python -c $(codeblock "
>> import pathlib
>> for _ in range(10):
>> print('this is a demo, bear with me')
>> ")
>> }
>>
>> Or more recently I found that this is a low-impact workaround:
>>
>> mybashfunc(){
>> python -c "if 1:
>> import pathlib
>> for _ in range(10):
>> print('this is a demo, bear with me')
>> "
>> }
>>
>> But as a certain Python dev may say: "There must be a better way."
>>
>> Would there be any downside to the Python CLI automatically dedenting the
>> input string given to -c? I can't think of any case off the top of my head
>> where it would make a previously valid program invalid. Unless I'm missing
>> something this would strictly make previously invalid strings valid.
>>
>> Thoughts?
>>
>> You can solve this with a small module. I named mine run_dedent
>>
>> mkdir -p run_dedent
>> echo pass >run_dedent/__init__.py
>> cat <run_dedent/__main__.py
>> import sys
>> import textwrap
>> cmd = textwrap.dedent(sys.argv[1])
>> exec(cmd)
>> EOF
>>
>> python3 -m run_dedent "
>> import sys
>> print(sys.version_info)
>> "
>>
>> Barry
>>
>>
>>
>> --
>> -Dr. Jon Crall (him)
>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to 
>> python-ideas-leave@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at 
>> https://mail.python.org/archives/list/python-ideas@python.org/message/6SYN2MQIP5DTF6INZ4SP2YKLF6P4VJOW/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
> --
> -Dr. Jon Crall (him)
>
>

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2GGM2J7D2MUMPKOEM45ZQEWRHI2QMN6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto dedent -c arguments.

[Jonathan Crall]

There's no question that there are lots of ways you can work around the
issue (I think the `if 1:` method is the easiest and most portable, but it
still is boilerplate which can be confusing if you don't know why it's
there). The question is: would enough people benefit from this to add the
feature to CPython?


On Sun, Apr 9, 2023 at 4:09 PM Barry Scott  wrote:

>
> On 04/04/2023 15:18, Jonathan Crall wrote:
>
> I have what I think is a fairly low impact quality of life improvement to
> suggest for the python CLI.
>
> When I'm not working in Python I tend to be working in bash. But often I
> want to break out and do something quick in Python. I find the `python -c `
> CLI very useful for this. For one liners it's perfect. E.g.
>
> NEW_VAR=$(python -c "import pathlib;
> print(pathlib.Path('$MYVAR').parent.parent)")
>
> And even if I want to do something multi-line it's pretty easy
>
> NEW_VAR=$(python -c "
> import pathlib
> for _ in range(10):
> print('this is a demo, bear with me')
> ")
>
> But the problem is when I'm writing bash inside a function or some other
> nested code, I would like to have nice indentation in my bash file, but if
> I write something like this:
>
> mybashfunc(){
> python -c "
> import pathlib
> for _ in range(10):
> print('this is a demo, bear with me')
> "
> }
>
> I get `IndentationError: unexpected indent`.
>
> This means I have to write the function ugly like this:
>
> mybashfunc(){
> python -c "
> import pathlib
> for _ in range(10):
> print('this is a demo, bear with me')
> "
> }
>
> Or use a helper function like this:
>
> codeblock()
> {
> __doc__='
> copy-pastable implementation
> Prevents indentation errors in bash
> '
> echo "$1" | python -c "import sys; from textwrap import dedent;
> print(dedent(sys.stdin.read()).strip('\n'))"
> }
>
> mybashfunc(){
> python -c $(codeblock "
> import pathlib
> for _ in range(10):
> print('this is a demo, bear with me')
> ")
> }
>
> Or more recently I found that this is a low-impact workaround:
>
> mybashfunc(){
> python -c "if 1:
> import pathlib
> for _ in range(10):
> print('this is a demo, bear with me')
> "
> }
>
> But as a certain Python dev may say: "There must be a better way."
>
> Would there be any downside to the Python CLI automatically dedenting the
> input string given to -c? I can't think of any case off the top of my head
> where it would make a previously valid program invalid. Unless I'm missing
> something this would strictly make previously invalid strings valid.
>
> Thoughts?
>
> You can solve this with a small module. I named mine run_dedent
>
> mkdir -p run_dedent
> echo pass >run_dedent/__init__.py
> cat <run_dedent/__main__.py
> import sys
> import textwrap
> cmd = textwrap.dedent(sys.argv[1])
> exec(cmd)
> EOF
>
> python3 -m run_dedent "
> import sys
> print(sys.version_info)
> "
>
> Barry
>
>
>
> --
> -Dr. Jon Crall (him)
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to 
> python-ideas-leave@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/6SYN2MQIP5DTF6INZ4SP2YKLF6P4VJOW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SEEFXCSRLIDETQFYKULS6EVK3U4AXKSF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Auto dedent -c arguments.

[Jonathan Crall]

I have what I think is a fairly low impact quality of life improvement to
suggest for the python CLI.

When I'm not working in Python I tend to be working in bash. But often I
want to break out and do something quick in Python. I find the `python -c `
CLI very useful for this. For one liners it's perfect. E.g.

NEW_VAR=$(python -c "import pathlib;
print(pathlib.Path('$MYVAR').parent.parent)")

And even if I want to do something multi-line it's pretty easy

NEW_VAR=$(python -c "
import pathlib
for _ in range(10):
print('this is a demo, bear with me')
")

But the problem is when I'm writing bash inside a function or some other
nested code, I would like to have nice indentation in my bash file, but if
I write something like this:

mybashfunc(){
python -c "
import pathlib
for _ in range(10):
print('this is a demo, bear with me')
"
}

I get `IndentationError: unexpected indent`.

This means I have to write the function ugly like this:

mybashfunc(){
python -c "
import pathlib
for _ in range(10):
print('this is a demo, bear with me')
"
}

Or use a helper function like this:

codeblock()
{
__doc__='
copy-pastable implementation
Prevents indentation errors in bash
'
echo "$1" | python -c "import sys; from textwrap import dedent;
print(dedent(sys.stdin.read()).strip('\n'))"
}

mybashfunc(){
python -c $(codeblock "
import pathlib
for _ in range(10):
print('this is a demo, bear with me')
")
}

Or more recently I found that this is a low-impact workaround:

mybashfunc(){
python -c "if 1:
import pathlib
for _ in range(10):
print('this is a demo, bear with me')
"
}

But as a certain Python dev may say: "There must be a better way."

Would there be any downside to the Python CLI automatically dedenting the
input string given to -c? I can't think of any case off the top of my head
where it would make a previously valid program invalid. Unless I'm missing
something this would strictly make previously invalid strings valid.

Thoughts?

-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6SYN2MQIP5DTF6INZ4SP2YKLF6P4VJOW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "Curated" package repo?

[Jonathan Crall]

I like the idea of a vetted package index that pip can point to. The more I
think about it, the more I think that it needs some sort of peer review
system as the barrier to entry, and my thoughts to to establishing some
DeSci DAO that could distribute the peer review of packages amongst a set
of trusted maintainers while also being a mechanism to add new trusted
maintainers to the peer-review pool. Peer reviewers could be funded via a
fee to submit a package for publishing. There are a lot of open questions
of how this would be done correctly - or even if this is necessary - but I
think to achieve a scalable, funded, decentralized, and trustworthy package
index a DAO makes some amount of sense.



On Tue, Jul 4, 2023 at 8:52 PM Dom Grigonis  wrote:

> To have 4) would be great. 2) could be a temporary test pilot to get some
> ideas when/if 4) was to be implemented.
>
> Starting point for 2) could be simple (?). E.g. 4 parts:
>   1. Features & integration
>   2. Performance - set-up automatic benchmarking via CI
>   3. Status (stack hits, user base, open issues, lines of code, update
> frequency & similar)
>   4. User rating (wander if up/down voting or star rating can be somehow
> implemented in github)
>
> Author collects inputs from this e-mail group or via opened issue and
> collates results. Once the comparison tables are there package authors can
> then issue PR themselves or next interested user does the update.
>
> Could be a simple way to gauge variables before committing to 3) or 4).
>
> On 5 Jul 2023, at 03:21, Christopher Barker  wrote:
>
> Stating a new thread with a correct title.
>
> On 2 Jul 2023, at 10:12, Paul Moore  wrote:
>
> Unfortunately, too much of this discussion is framed as “someone should”,
>> or “it would be good if”. No-one is saying “I will”. Naming groups, like
>> “the PyPA should” doesn’t help either - groups don’t do things, people do.
>> Who in the PyPA? Me? Nope, sorry, I don’t have the time or interest - I’d
>> *use* a curated index, I sure as heck couldn’t *create* one.
>
>
> Well, I started this topic, and I don't *think* I ever wrote "someone
> should", and I certainly didn't write "PyPa should".
>
> But whatever I or anyone else wrote, my intention was to discuss what
> might be done to address what I think is a real problem/limitation in the
> discoverability of useful packages for Python.
>
> And I think of it not so much as "someone should" but as "it would be nice
> to have".
>
> Of course, any of these ideas would take a lot of work to implement --
> and  even though there are a lot of folks, on this list  and elsewhere,
> that would like to help, I don't think any substantial open-source project
> has gotten anywhere without a concerted effort by a very small group (often
> just 1) of people doing a lot of work to get it to a useful state before a
> larger group can contribute. So I"m fully aware that nothings going to
> happen unless *someone* really puts the work in up front. That someone
> *might* be me, but I'm really good at over-committing myself, and not so
> great at keeping my nose to the grindstone, so 
>
> And I think this particular problem calls for a solution that would have
> to be pretty well established before reaching critical mass to actually be
> useful -- after all, we already have PyPi -- why go anywhere else that is
> less comprehensive?
>
> All that being said, it's still worth having a conversation about what a
> good solution might look like -- there are a lot of options, and hashing
> out some of the ideas might inspire someone to rise to the occasion.
>
> The :problem", as I see it.
>
>  - The Python standard library is not, and will never be fully
> comprehensive -- most projects require *some* third party packages.
>  - There are a LOT of packages available on PyPi -- with a very wide range
> of usefulness, quality and maintenance -- everything from widely used
> packages with a huge community (e.g. numpy) to packages that are release
> 0.0.1, and never seen an update, and may not even work.
>
> So the odds that there's a package that does what you need are good, but
> it can be pretty hard to find them sometimes -- and can be a fair bit
> of work to sift through to find the good ones -- and many folks don't feel
> qualified to do so.
>
> This can result in two opposite consequences:
>
> 1) People using a package that really isn't reliable or maintained (or not
> supported on all platforms, or ..) and getting stuck with it (I've had that
> on some of my projects -- I'm pretty careful, but not everyone on my team
> is)
>
> 2) People writing their own code - wasting time, and maybe not getting a
> very good solution either. I've also had that problem on my projects...
>
> To avoid this -- SOME way for folks to find packages that have at least
> has some level of vetting would be good -- exactly what level of vetting,
> is a very open question, but I think "even a little" could be very helpful.
>
> A few ideas that have come up 

[Python-ideas] Re: "Curated" package repo?

[Jonathan Crall]

> On Mon, Jul 24, 2023 at 10:04 AM Chris Angelico  wrote:
> can you tell me what this vetting procedure proves that isn't already
proven by mere popularity itself?

I think that's what this thread is trying to discuss. Do I have the exact
perfect implementation? No. But I imagine it to be akin to peer-review. I
can't prove this, but I think it adds signal that complements popularity.

> And are you also saying that packages should be *removed* from this
curated list?

Yes, absolutely. If packages fall out of maintenance, are deprecated, or
end-of-life, they should no longer be this curated list. I imagine the
mechanism would be a series of snapshots of what the state of the list is
at a particular point in time. What is the mechanism for determining the
trigger to remove a package? No clue right now.  There are a lot of
problems - most of them social - that need to be sorted out to make a
*useful* curated package list a reality. I don't claim to have the answers,
but I'm willing to participate in discussion to find them.

> More robust in what way?

It's curated by people I trust more than the average bear.

> If not, how is it different from "yet another collection"?

We are discussing details about it here instead of just posting what we
think should be there on github right now. I'm putting a bit of trust in
this group to find a way to do that. I do think that separating the
opinions of experts (however we choose to define that group - but I hope
you agree that it should be possible to find *some *reasonable definition)
as an auxiliary re-ranking on top of popularity is a good differentiator.



On Tue, Jul 25, 2023 at 1:31 AM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> George Fischhof writes:
>
> [For heaven's sake, trim!  You expressed your ideas very clearly, the
> quote adds little to them.]
>
>  > it has got to my mind that even just grouping similar / same goal
>  > packages could help the current situation.
>
> This is a good idea.  I doubt it reduces the problem compared to the
> review site or the curation very much: some poor rodent(s) still gotta
> put the dinger on the feline.
>
> However, in designing those pages, we could explicitly ask for names
> of similar packages and recommendations for use cases where an
> alternative package might be preferred, and provide links to the
> review pages for those packages that are mentioned in the response.
> We can also provide suggestions based on comparisons other users have
> made.  (Hopefully there won't be too many comparisons like "this
> package is the numpy of its category" -- that's hard to parse!)
>
>  > Additionally perhaps the users could give relative valuation,
>
> Not sure asking for rankings is a great idea, globally valid rankings
> are rare -- ask any heavy numpy user who occasionally uses the sum
> builtin on lists.
>
>  > for example there are A, B, C, D similar packages, users could say:
>  > I tried out A and B, and found that A is better then B, and could
>  > have some valuation categories: simple, easy, powerful etc. This
>  > would show for example that package A is simple, but B is more
>  > powerful
>
> These tags would be useful.  I think the explanation needs to be
> considered carefully, because absolutes don't really exist, and if
> you're comparing to the class, you want to know which packages the
> reviewer is comparing to.  I'm not sure many users would go to the
> trouble of providing full rankings, even for the packages they've
> mentioned.  Worth a try though!
>
> Steve
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/MGYK3EEEYKISRQMFJY7U73ID6VBQY4AT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VOIGDXADBJIL2GKIYJE6STVY3XQXQFDG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "Curated" package repo?

[Jonathan Crall]

If popular packages weren't favored that would be a problem. Popularity
should be correlated with "trustworthiness" or whatever the metric this
curated repo seeks to maximize. I think the important thing is that the
packages are both popular and have passed some sort of vetting procedure.

For instance, for a very long time Python2 was far more popular than
Python3, but any expert in the field would encourage users to move to
Python3 sooner rather than later. Python2 is popular, but it wouldn't have
made the cut on some expert-curated list.

So it helps in that it reranks popular packages (and also excludes some)
for those who want to adopt a more strict security / reliability posture.

By no means do I think this would replace pypi as the de-facto packaging
repository. Its low barrier to entry is extremely important for a thriving
community, but I also wouldn't mind having something a bit more robust.

I also think this project would have to careful not to become yet another
"awsome-python-package" collection. Those certainly have value, but based
on the initial proposal, I'm interested in something a tad more robust.

On Mon, Jul 24, 2023 at 8:55 AM Chris Angelico  wrote:

> On Mon, 24 Jul 2023 at 21:02, James Addison via Python-ideas
>  wrote:
> > ... some thoughts on how to build a scalable, resilient trust network
> based on user ratings; I can't guarantee that it'll change your opinion,
> though!
> >
>
> This still has the fundamental problems of any sort of user rating
> system: popular packages are inherently favoured. And we can already
> get a list of popular packages, because download stats are available.
> So how would this scheme help?
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/LU6BFQGNCMZZVESCUUCPSVKWPKJEJB7H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WRF6VSZ47KMGJI3PZEWBQCHFFGYE7AJ2/
Code of Conduct: http://python.org/psf/codeofconduct/