Re: [Python-ideas] Add "default" kwarg to list.pop()

[Michel Desmoulin]

+1 one this.

And a list.get method has well, with a default value.

Le 31/10/2018 à 00:44, Giampaolo Rodola' a écrit :

Sorry in advance if this has been proposed in the past but I couldn't
find anything on python-ideas:

 >>> l = []
 >>> l.pop(default=1)
1

FWIW my use case consists in reading entries from /proc/diskstats where
lines can have a variable number of fields depending on the kernel version:
https://github.com/giampaolo/psutil/issues/1354#issuecomment-434495870
https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cacb2127/psutil/_pslinux.py#L1068
As such it would be convenient to set missing fields to 0 as "reads =
fields.pop(default=0)" instead of catching IndexError every time. Extra:
for consistency set.pop() should probably have the same.

Thoughts?

--
Giampaolo - http://grodola.blogspot.com


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JS’ governance model is worth inspecting

[Michel Desmoulin]

Le 22/09/2018 à 20:27, James Lu a écrit :
> > To my mind, there is one very big reason we should be cautious about
> > adopting JS language-design policies, namely, that they have led to a
> > very, very poorly designed language.  No doubt a good deal of that is
> > baggage from early stages in which JS had a poor to nonexistent language
> > design governance model.  Nonetheless, the failure of JS to fix its
> > numerous fundamental flaws, and especially the rapid feature churn in
> > recent years, suggests to me that their model should be viewed with
> > skepticism.
> I disagree. The language is often very flexible and effective in its
domains. I don’t know what you mean by “rapid feature churn”, churn
usually means existing features are superseded by newer ones- this isn’t
the case.
>
> JS is much more nuanced than it appears on the surface. It’s
understandable that those with only a glossing of JS look down on it,
because JS really was a primitive language a few years ago.
>
> You can learn about JS in depth with the poorly-named “You don’t know
JS” free online book.
>

I worked with JS for the last 10 years, and I agree that "we should be
cautious about
adopting JS language-design policies", particularly about the fact they
completly ignored readability in their concerns.

But still, using the old JS baggages to justify we reject what they are
doing currently is not a good argument:

  - they can't break the whole Web so deprecation is very hard. Python 2
=> 3 should make us understand that. Yes it sucks you can still declare
a variable global by default. It also sucked we had to rewrite most good
python modules during the last decade.
  - the new JS features have been so far a good fit for the language and
overall made it better.
  - fast pace evolution is only for the JS ecosystem (and I agree it's
terrible). But the spec and implementations
have been very reasonable in their progress.

Now it's hard to know if it's because of the design policy or in spite
of it. But while I still dislike JS, it IS a vastly better language that
it used to be and we should not disregard the design policies because of
these particular issues.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

[Michel Desmoulin]

Le 19/09/2018 à 15:28, Chris Angelico a écrit :
> On Wed, Sep 19, 2018 at 11:23 PM Michel Desmoulin
>  wrote:
>> - A is telling B this is a bad idea. It should be easy to tell if the
>> person is experienced or not. You probably don't want to interact the
>> same way with Victor and Yury, that have done numerous contributions to
>> the Python core, and me, that is just a regular Python dev and don't
>> know how the implementation work.
> 
> Hmm, I'm not sure about this. Shouldn't a person's arguments be
> assessed on their own merit, rather than "oh, so-and-so said it so it
> must be right"?

"Merit" is something hard to evaluate, having context help. If somebody
comes and says "this is hard to implement so I doubt it will pass", Tim
Peters does know better than the average Joe.

If somebody says, "I advise you to do things the other way around, it
works better on this mailing list". You will consider the advice more
strongly if the author has been on the list 10 years or 10 days.

Above all, if 2 people have opposite views, and that they both make
sense, having the context of who they are helping.

It's the same has if somebody gives you health advice. You do want to
listen to everybody, but it's nice to know who is a doctor, and who is a
somebody who repeats Facebook posts. It helps to decide.

> 
> But if you want to research the people who are posting, you're welcome
> to do that. The list of core dev experts is on the devguide:
> 
> https://devguide.python.org/experts/
> 
> Translating those usernames back into real names would be done via BPO, I 
> think.

This is a good summary of the problem with the list: you can do anything
you want, but it cost you time and effort. And since you have many
things to do, cumulatively, it's a lot of time and effort.

I read all the posts and answered 2 mails on the list today. It took me
40 minutes. And I have been on the list for a long time so I know how
the whole thing works so I'm pretty fast at doing this.

Who can spend a lot of time every day, and yet feels to be just barely
part of the discussion ? Who will take the time to do things right ? And
among those few people, couldn't they do more good things if we'd save
them time and energy ?

Let's make the tool work for the community, and not against it.

I agree that the mailing list is a great format for things like Python-dev.

However, it's not a good fit for Python-idea: we have reached the limit
of it for a long time. Most of the real decisions are actually taken
outside of it, with more direct channels in the small groups of
contributors. It slows down the decision process and it waste a lot of
good will.

> 
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

[Michel Desmoulin]

Le 19/09/2018 à 00:37, James Lu a écrit :
>> Is that really an issue here? I personally haven't seen threads where
>> Brett tried to stop an active discussion, but people ignored him and
>> kept fighting.
> Not personally with Brett, but I have seen multiple people try to stop the 
> “reword or remove beautiful is better than ugly in Zen of Python.” The 
> discussion was going in circles and evolved into attacking each other’s use 
> of logical fallacies. 
> 
> Other than that, my biggest issues with the current mailing system are:
> 
> * There’s no way to keep a updated proposal of your own- if you decide to 
> change your proposal, you have to communicate the change. Then, if you want 
> to find the authoritative current copy, since you might’ve forgotten or you 
> want to join he current discussion, then you have to dig through  the emails 
> and recursively apply the proposed change. It’s just easier if people can 
> have one proposal they can edit themselves.
>   * I’ve seen experienced people get confused about what was the current 
> proposal because they were replying to older emails or they didn’t see the 
> email with the clear examples.
> * The mailing list is frankly obscure. Python community leaders and package 
> maintainers often are not aware or do not participate in Python-ideas. Not 
> many people know how to use or navigate a mailing list.
>   * No one really promotes the mailing list, you have to go out of your way 
> to find where new features are proposed. 
>   * Higher discoverability means more people can participate, providing their 
> own use cases or voting (I mean using like or dislike measures, consensus 
> should still be how things are approved) go out of their way to find so they 
> can propose something. Instead, I envision a forum where people can read and 
> give their 2 cents about what features they might like to see or might not 
> want to see. 
>* More people means instead of having to make decisions from sometimes 
> subjective personal experience, we can make decisions with confidence in what 
> other Python devs want. 
> 
> Since potential proposers will find it easier to navigate a GUI forum, they 
> can read previous discussions to understand the reasoning, precedent behind 
> rejected and successful features. People proposing things that have already 
> been rejected before can be directed to open a subtopic on the older 
> discussion. 

+1 except for visibility

I have been on this list for years and those issues have been a big
problem ever since.

But I agree with Antoine, quantity is not the problem. Quality is.

However having no way to moderate efficiently means nobody does it,
which means quality goes down. Since you have no way to identify who is
who anyway, you can't know if the person telling you that you are out of
line is an experienced member of the community or a newcomer with a lot
of energy.

Another things is that we keep having the same debates over and over. If
you had the same duplication in code, it would never pass code reviews.
The problem is looking up something, or making a reference to something
is really hard on the list.

A few scenario that seem important to me that are badly handled by this
tool:

- Person A is making a long constructive argument, and person B arrives,
doesn't read anything, and make arguments against things that have been
answered. It should be easy for somebody to link to the answers to this.

- Somebody is making a proposal that has been already discussed and
rejected several times. It should be easy to link to the discussions and
conclusions about this. Even if the goal is to start the debate over
again, at least we start ahead.

- A is telling B this is a bad idea. It should be easy to tell if the
person is experienced or not. You probably don't want to interact the
same way with Victor and Yury, that have done numerous contributions to
the Python core, and me, that is just a regular Python dev and don't
know how the implementation work.

- somebody wants to make a proposal. It should be easy to search if
similar proposals already have been made, and read __ a summary __ of
what happened. The bar to write a PEP is to high to serve that purpose:
most proposal don't ever leave the list.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Pre-conditions and post-conditions

[Michel Desmoulin]

Isn't the purpose of "assert" to be able to do design by contract ?

assert test, "error message is the test fail"

I mean, you just write your test, dev get a feedback on problems, and
prod can remove all assert using -o.

What more do you need ?

Le 15/08/2018 à 23:06, Marko Ristin-Kaufmann a écrit :
> Hi,
> 
> I would be very interested to bring design-by-contract into python 3. I
> find design-by-contract particularly interesting and indispensable for
> larger projects and automatic generation of unit tests.
> 
> I looked at some of the packages found on pypi and also we rolled our
> own solution (https://github.com/Parquery/icontract/
> ). I also looked into
> https://www.python.org/dev/peps/pep-0316/
> .
> 
> However, all the current solutions seem quite clunky to me. The
> decorators involve an unnecessary computational overhead and the
> implementation of icontract became quite tricky once we wanted to get
> the default values of the decorated function.
> 
> Could somebody update me on the state of the discussion on this matter?
> 
> I'm very grateful for any feedback on this!
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Keyword only argument on function call

[Michel Desmoulin]

Le 06/09/2018 à 03:15, Anders Hovmöller a écrit :
> I have a working implementation for a new syntax which would make using 
> keyword arguments a lot nicer. Wouldn't it be awesome if instead of:
> 
>   foo(a=a, b=b, c=c, d=3, e=e)
> 
> we could just write:
> 
>   foo(*, a, b, c, d=3, e)
> 

It will make code harder to read.

Indeed, now your brain has to make the distinction between:

foo(a, *, b, c)

and:

foo(a, b, *, c)

Which is very subtle, yet not at all the same thing.

All in all, this means:

- you have to stop to get the meaning of this. Scanning the lines
doesn't work anymore.
- this is a great opportunity for mistakes, and hence bugs.
- the combination of the two makes bugs that are hard to spot and fix.

-1



> and it would mean the exact same thing? This would not just be shorter but 
> would create an incentive for consistent naming across the code base. 
> 
> So the idea is to generalize the * keyword only marker from function to also 
> have the same meaning at the call site: everything after * is a kwarg. With 
> this feature we can now simplify keyword arguments making them more readable 
> and concise. (This syntax does not conflict with existing Python code.)
> 
> The full PEP-style suggestion is here: 
> https://gist.github.com/boxed/f72221e7e77370be3e5703087c1ba54d
> 
> I have also written an analysis tool you can use on your code base to see 
> what kind of impact this suggestion might have. It's available at 
> https://gist.github.com/boxed/610b2ba73066c96e9781aed7c0c0b25c . The results 
> for django and twisted are posted as comments to the gist. 
> 
> We've run this on our two big code bases at work (both around 250kloc 
> excluding comments and blank lines). The results show that ~30% of all 
> arguments would benefit from this syntax.
> 
> Me and my colleague Johan Lübcke have also written an implementation that is 
> available at: https://github.com/boxed/cpython 
> 
> / Anders Hovmöller
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting dedicated overloadable boolean operators

[Michel Desmoulin]

Adding one operator is hard in Python.

Adding 4 operators, just for the sake of a bit of syntaxic suggar for
DSL based projects is never going to fly.

And I say that as a long time SQLA user.

Le 03/08/2018 à 19:46, Todd a écrit :
> Coming back to the previous discussion about a new set of overloadable
> boolean operators [1], I have an idea for overloadable boolean operators
> that I think might work.  The idea would be to define four new operators
> that take two inputs and return a boolean result based on them.  This
> behavior can be overridden in appropriate dunder methods.  These
> operators would have similar precedence to existing logical operators. 
> The operators would be:
> 
> bNOT - boolean "not"
> bAND - boolean "and"
> bOR - boolean "or"
> bXOR - boolean "xor"
> 
> With corresponding dunder methods:
> 
> __bNOT__ and _rbNOT__ (or __r_bNOT__)
> __bAND__ and _rbAND__ (or __r_bAND__)
> __bOR__ and _rbOR__ (or __r_bOR__)
> __bXOR__ and _rbXOR__ (or __r_bXOR__)
> 
> The basic idea is that the "b" is short for "boolean", and we change the
> rest of the operator to upercase to avoid confusions with the existing
> operators.  I think these operators would be preferably to the proposals
> so far (see [1] again) for a few reasons:
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] As-do statements/anonymous blocks in python

[Michel Desmoulin]

I like the concept, and I several times wished I could have had a
reference to the block of code in a `with` clause.

However, it is unlikely to happen:

1 - Guido restricted lambda to one expression, and this would just be a
way around that

2 - This will be tempting to use for callbacks and chaining things,
leading to the nesting hell we tried very carefully to avoid fom other
languages. The example for sorted is also kinda twisted.

3 - Most of the example (and you show it yourself with flask) are
already possible with a more verbose syntaxe based on decorators. Example:


@atexit.register
def _():
print('Goodbye')

4 - introducing a new keyword is the hardest thing you can ever ask on
this list.



Le 25/07/2018 à 19:07, James Lu a écrit :
> I'm open to any changes or criticism.
> 
> ```
> import atexit
> as atexit.register:
>     # ...do various cleanup tasks...
>     print('Goodbye')
> 
> # is approximately equivalent to => 
> import atexit
> def _():
>     # ...do various cleanup tasks...
>     print('Goodbye')
> atexit.register(_)
> 
> # flask example
> @app.route("/")
> def hello():
>     return "Hello World!"
> # is approximately equivalent to =>
> as app.route('/'):
>     return "Hello World!"
> 
> @app.route('/user/')
> def show_user_profile(username):
>     # show the user profile for that user
>     return 'User %s' % username
> 
> as app.route('/user/') do username:
>     return "Hello World!"
> 
> def print_sorted(iterable, block):
>     sorted(iterable, )
> 
> l = [1, 2, 3, 4, 'spam']
> 
> as l.sort(key=%) do obj:
>     return str(obj)
> 
> # multiple arguments
> as spam do a, b:
>     ...
> ```
> ## `%`  function call syntax
> Calling a function with a single percent in place of an argument creates
> a new function. 
> 
> ```
> lumberjack(15, %)
> # is equivalent to the expression
> lambda x: lumberjack(15, %)
> ```
> Using `*` instead of `%` could also be possible.
> 
> ```
> import threading, time
> def interval(seconds, block):
>     def target():
>         while True:
>             time.sleep(seconds)
>             if block():  # stop looping if block returns True
>                 break
>     threading.Thread(target=target).start()
> 
> as interval(5, %):
>    print("chirp")
> # => chirp every 5 seconds on a seperate thread
> as threading.Timer(5, %):
>    print("hi")
> # => say "hi" in 5 seconds
> ```
> 
> ## `^` currying function definition syntax?
> I'm not sure this is necessary or a good idea.
> ```
> def interval(seconds, ^block):
>     def target():
>         while True:
>             time.sleep(seconds)
>             if block():  # stop looping if block returns True
>                 break
>     threading.Thread(target=target).start()
> # is aprroximately equivalent to
> def interval(seconds, block=None):
>     def inner(block):
>         def target():
>             while True:
>                 time.sleep(seconds)
>                 if block():
>                     break
>         threading.Thread(target=target).start()
>     if block == None:
>         def outer(block):
>             return inner(block)
>     else:
>         return inner(block)
> 
> as interval(5):
>     print('chirp')
> # equivalent to 
> interval(5)(lambda: print('chirp'))
> ```
> ### Lazy evaluation of chained `%` calls?
> This would allow things like:
> ```
> start_on_new_thread = threading.Thread(target=%).start()
> def bong():
>     while True:
>         time.sleep(6*60)
>         print('BONG')
> start_on_new_thread(bong)
> # alternatively
> as start_on_new_thread:
>     while True:
>         time.sleep(6*60)
>         print('BONG')
> ```
> 
> ## As-do statements in classes
> ```
> class M():
>      def __init__(self):
>          self.time = 5
>      as interval(self.time, %):
>          print('chirp')
> ```
> 
> I'm not sure if this should be valid, and I'd like the community's input
> on when as-do statements should be bound when as-do.
> 
> ## Proposed Type Hinting Syntax
> ```
> as app.route('/user/') do (username: str): 
>     return "Hello World!"
> ```
> 
> Alternatives:
> ```
> as app.route('/user/') do username: str%
>     return "Hello World!"
> # like objective-c
> as app.route('/user/') do username: str^
>     return "Hello World!"
> # like coffeescript
> as app.route('/user/') do username: str ->
>     return "Hello World!"
> ```
> 
> I’m not totally sure of practical uses, but I’m imagining it would make
> passing a function to another function much more convenient. In React,
> you pass an argument called `render` to a `FlatList`, and `render`
> renders an item of the list. `FlatList` is a scrollable list that
> handles unloading off-screen items and loading items that will appear in
> the scroll box soon.
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

Re: [Python-ideas] A better (simpler) approach to PEP 505

[Michel Desmoulin]

Le 23/07/2018 à 17:12, David Mertz a écrit :
> The need addressed by PEP 505 is real; it's also MUCH more niche and
> uncommon than something that would merit new syntax.  Moreover, the
> actual legitimate purpose served by the PEP 505 syntax is easily served
> by existing Python simply by using a wrapper class.
>
> Here is a way of solving the "deep attribute access to messy data"
> problem that is:
>
> (1) Much more explicit
> (2) Requires no change in syntax
> (3) Will not be a bug magnet
> (4) Inasmuch as there are semantic traps, they are announced by the use
> of a class whose documentation would be pointed to for readers
>
> The API that could be useful might be something like this:
>
> In [1]: from none_aware import NoneAware
> In [2]: from types import SimpleNamespace
> In [3]: foo = SimpleNamespace()
> In [4]: foo.bar = SimpleNamespace()
> In [5]: foo.bar.baz = SimpleNamespace()
> In [6]: foo.bar.baz.blat = 42
> In [7]: NoneAware(foo).bar.blim
> Out[7]: 
> In [8]: NoneAware(foo).bar.blim.unbox()
> In [9]: NoneAware(foo).bar.baz.blat.unbox()
> Out[9]: 42
> In [10]: NoneAware(foo).bar.baz.blat
> Out[10]: 
> In [11]: NoneAware(foo).bar.baz.flam.unbox()
> In [12]: NoneAware(foo).bar.baz.flam
> Out[12]: 
This has existed as libs for a while:

https://github.com/ekampf/pymaybe


It's interest is much more limited than in Haskell because in Python,
calls are not lazy, objects are dynamic and function calls are expensive.


Take the following:

foo().bar[0]

Doing it safely would be:

val = foo()

try:
val = val.bar
except AttributeError:
val = None
else:
try:
val = val[0]
except KeyError:
val = None

Clearly we see that the operation goes up to the end if everything goes
right. Otherwise, it jumps to the default value.

With the operator proposal:

val = foo()?.bar?[0]

The "?" operator is like "or" / "and", and it will shotcircuit as well.

With the try / else proposal:

val = try foo().bar[0] else None

This is supposed to transform this ast to into the first try/except
form. So it shortcircuit as well.

But with your proposal, it becomes:

val = maybe(foo()).bar[0].or_else(None)

Which means

foo() # call

maybe(foo()) # call

maybe(foo()).__getattr__('bar') # call

maybe(foo()).__getattr__('bar').__getitem__(0) # call

maybe(foo()).__getattr__('bar').__getitem__(0).or_else(None) # call

There is no shortcircuit, you get 5 calls in all cases, plus the maybe
object proxing the call to the underlying value every time. So that's 7
calls, added to the try/excepts you still have behind the scene.

Plus, you don't get the same calls depending of the values, but the
switch is implicit and behind the maybe object: no linter can save you
from a typo.

So yes, it works right now. But it's a suboptimal solution.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

[Michel Desmoulin]

Proposal a __very__ dumbed down "try" expression


variable = try Expression as not sentinel [else default]

The semantic is:

1 - try the expression. If any call (from a normal method, or a
__dunder__ one) returns the sentinel, we shortcircuit.

2 - if we shortcircuit, the variable will be set do the default value.
If no default is provided, the variable is set to the sentinel.

3 - if we don't shortcircuit, the returned value is the one from the
expression.


Example
==


value = try foo.bar[0] as not None

If anything returns None, we shortcircuit and value will end up being
None. Otherwise value contains the result of foo.bar[0].

value = try foo.bar[0] as not True

If anything returns True, we shortcircuit and value will end being True.
Otherwise value contains the result of foo.bar[0].

value = try foo.bar[0] as not None else "foo"

If anything returns None, we shortcircuit and value will end up being
'foo'. Otherwise value contains the result of foo.bar[0].


value = try foo.bar[0] as not True else "foo"

If anything returns True, we shortcircuit and value will end up being
'foo'. Otherwise value contains the result of foo.bar[0].


This as several benefits:
=

- Does not introduce new keywords or operators

- Read pretty much as ordinary Python

- Allow for inline try, as suggested before several times on
python-idea, but restrict it to a very narrow subset of what it can do.
This will end the debate about inline try/except, while sill providing a
sane subsets of use cases it would have provided.

- more flexible than ? as it allows to choose what to choose the
sentinel and the default value instead of just None.

- the expression is untouched. It's surrounded by the rest, so it still
reads as the expression would without try / as.

- avoding "except", exception names and ":" make sure we don't confuse
this try expression with the regular try/except statements. try would
then be a very restricted version of it, like lambda is for def.


If you don't like these keywords
===

We can opt for something different:

value = try Expression as not sentinel [or default]
value = try Expression if not sentinel [else default]
value = try Expression with not sentinel [else default]

The proposal is more about the structure try (stuff), guard against a
sentinel, and optionally return a default value.

Besides, I don't know what the grammar would allow.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] grouping / dict of lists

[Michel Desmoulin]

Counter also consider any missing key has the value "0".

With the constructor (accepting any iterable) and the most_common(n),
it's just a very set of features if you need to count anything.

Le 13/07/2018 à 19:45, Michael Selik a écrit :
> On Mon, Jul 2, 2018 at 8:49 AM Chris Barker  > wrote:
> 
> On Fri, Jun 29, 2018 at 11:25 PM, Guido van Rossum  > wrote:
>  
> 
> Hm, this actually feels heavier to me. But then again I never
> liked or understood the need for Counter --
> 
> 
> actually, me neither -- and partly because it's too lightweight --
> that is, it's still a regular dict, and you pretty much have to know
> that to use it. That it, it provides a nice counting constructor,
> but after that, it's just a key:integer dict :-)
> 
> 
> Counter provides ``most_common`` which is often implemented
> inefficiently if written from scratch. People mistakenly use ``sorted``
> instead of ``heapq.nlargest``.
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow filtered dir built in

[Michel Desmoulin]

Le 14/06/2018 à 11:27, Steve Barnes a écrit :
> Currently when working with interactive sessions using the dir() or 
> dir(module) built in is incredibly useful for exploring what 
> functionality is available in a module. (Especially the regrettable 
> libraries or modules that add really valuable functionality but have no 
> or limited docstrings).
> 
> However I often find that when a module adds a lot of functions I need 
> to filter those entries to be able to find the one that I need, e.g.:
> 
>  >>> import mpmath
>  >>> dir(mpmath)  # This produces 390+ lines of output but
>  >>> for name in dir(mpmath):
> ...if 'sin' in name:
> ...print(name)  # gives me a mere 13 to consider as candidates
> 
> What I would really like to do is:
>  >>> dir(mpmath.*sin*)
> 
> However, I know that the interpreter will hit problems with one or more 
> operators being embedded in the module name.
> 
> What I would like to suggest is extending the dir built-in to allow an 
> optional filter parameter that takes fnmatch type wild card as an 
> optional filter. Then I could use:
> 
>  >>> dir(mpmath, "*sin*")
> 
> To narrow down the candidates.
> 
> Ideally, this could have a recursive variant that would also include 
> listing, (and filtering), any sub-packages.
> 

Fantastic idea. Would this make sense on var() too ? It's not exactly
the same usage context.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Give regex operations more sugar

[Michel Desmoulin]

Le 14/06/2018 à 07:29, Steven D'Aprano a écrit :
> On Wed, Jun 13, 2018 at 10:59:34PM +0200, Michel Desmoulin wrote:
> 
>>> Attaching an entire module to a type is probably worse than
>>> adding a slew of extra methods to the type.
>>>
>>
>> Not my point.
>>
>> str.re would not be the re module, just a namespace where to group all
>> regex related string methods.
> 
> That's what a module is :-)
> 
> How would this work? If I say:
> 
> "My string".re.match(...)
> 
> if str.re is "just a namespace" how will the match function know the 
> string it is to operate on?

There are a lot of ways to do that. One possible way:

import re

class re_proxy:

def __init__(self, string):
self.string = string

def match(self, pattern, flags):
return re.match(pattern, self.string, flags)

...

@property
def re(self):
return re_proxy(self)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Give regex operations more sugar

[Michel Desmoulin]

Le 13/06/2018 à 19:11, Mike Miller a écrit :
> 
> On 2018-06-13 06:33, Michel Desmoulin wrote:
>>
>> I often wished for findall and sub to be string methods, so +1 on that.
>>
> 
> Agreed, and there are a few string functions that could be extended (to
> take a sequence) to handle more cases that push folks to regex, perhaps
> earlier than they should.

str.replace come to mind. It's a annoying to have to chain it 5 times
while we could pass optionally a tuple.

several startswith() and endswith() require a loop, but we could make
them accept *args.

Also, we do have to saturate the str namespace with all the re
functions. We could decide to go for `str.re.stuff`.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Give regex operations more sugar

[Michel Desmoulin]

>> I suggest building all regex operations into the str class itself, as well
>> as a new syntax for regular expressions.
> 
>There are many different regular expression implementation (regex,
> re2). How to make ``s.search(pattern)`` work with all of them?
> 
You don't, they work stand alone anyway. Besides, nobody is proposing to
retire the re module either.

But if it's really important, you can make hooks to provide the
implementation like we did with breakpoint().

I really wish, however, than we separate the issue of adding the methods
on the str object, and making literals.

I know that literals are going to be rejected, "python is not perl", etc.

But the str methods are quite an interesting idea.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Give regex operations more sugar

[Michel Desmoulin]

Le 13/06/2018 à 13:06, Ken Hilton a écrit :
> Hi all,
> 
> Regexes are really useful in many places, and to me it's sad to see the
> builtin "re" module having to resort to requiring a source string as an
> argument. It would be much more elegant to simply do "s.search(pattern)"
> than "re.search(pattern, s)".
> I suggest building all regex operations into the str class itself, as
> well as a new syntax for regular expressions.
> 
> Thus a "findall" for any lowercase letter in a string would look like this:
> 
>     >>> "1a3c5e7g9i".findall(!%[a-z]%)
>     ['a', 'c', 'e', 'g', 'i']
> 
> A "findall" for any letter, case insensitive:
> 
>     >>> "1A3c5E7g9I".findall(!%[a-z]%i)
>     ['A', 'c', 'E', 'g', 'I']
> 
> A substitution of any letter for the string " WOOF WOOF ":
> 
>     >>> "1a3c5e7g9i".sub(!%[a-z]% WOOF WOOF %)
>     '1 WOOF WOOF 3 WOOF WOOF 5 WOOF WOOF 7 WOOF WOOF 9 WOOF WOOF '
> 
> A substitution of any letter, case insensitive, for the string "hovercraft":
> 
>     >>> "1A3c5E7g9I".sub(!%[a-z]%hovercraft%i)
>     '1hovercraft3hovercraft5hovercraft7hovercraft9hovercraft'
> 
>

I often wished for findall and sub to be string methods, so +1 on that.

But there is really no need for a literal. A string pattern is plenty.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

> 
> I still don't get it... If you have a framework you presumably have an
> entry point. Why can't you set up your policy in that entrypoint?  Why
> would a user attempt to change the policy at runtime (you haven't
> listed examples of libraries that do this)? 

Not at run time. But several libraries have several entry points, that
you must use in proper order, and setup correctly. Locking helps with
finding when you do an improper setup without reading all documentation
for all libs completely to make sure you didn't miss a corner case.

It will also help project owners because people will come to the bug
tracker saying "I have LockedPolicyError" when I do that, which is a lot
easier to solve than "I have an AttributeError on a None object".

 I see a lot of "I want to
> protect users from ..." arguments but I haven't yet seen "this and
> that happened in production and we haven't been able to debug what
> happened for a while".

It's like with asyncio.get_event_loop. We haven't seen the problem for a
a long time. Then eventually, people starting to use several loops in
several threads, and they said that it caused a problem.

It's the same here. I'm advising that we don't wait for people to have
the problem to solve it.

> 
> Do you handle cases when people install a blocking logging handler in
> their async application?  

We can't prevent that without a great cost. It's terribly complicated
problem, especially since logging is blocking and thread safe by design.

There is not even good documentation on the best practice on using
logging with asyncio. Most people have no idea how to do it (although in
my experience, most dev use the logging module incorrectly outside of
asyncio too). Twisted rewrote an entire logging system to solve that
problem.

Locking the policy is not a great cost. And it's about creating a user
friendly API.

Do you handle cases when a malfunctioning
> sys.excepthook is installed?  

When I override sys.excepthook, I take great care of backuping the old
hook, and calling it in my hook. And I really wish the stdlib had
something built in to do that cleanly, because I'm pretty sure most lib
overriding sys.excepthook all do that in a different way, if they do it
at all. Every time I use something to help with stack trace (coloring,
auto logging, etc), I have to read the source code to check they are
compatible. I should not have to do that.

What about cases when users accidentally
> import gevent somewhere in their asyncio application and it
> monkeypatches the 'socket' module (this is a real horror story, by the
> way)? 

Monkey patching is not officially supported by anything, anywhere, so
this argument is moot.

But it opens currently another door for my argument: currently the only
way to make sure nobody erase our policy is to monkey patch the function
to set the policy. Do we really want monkey patch to be the only
solution to this problem ?

 My point is that there are so many things that users can do
> that will break any framework, be it asyncio or django or trio.

Yes, eventually the question is how easily and cleanly can we provide a
solution to avoid the problem or help to debug it.

Django is a good example. I run sanity checks on common errors on your
model on startup to ease your life as a dev, and the life on the
maintainers and their bug tracker.

> 
> This sounds like "if something can happen it will happen" kind of
> thing, 

The reason I bring get_event_loop on the table is that I knew years ago
when I read that source code that it would come back to bite us. And I
said nothing because I assume the core devs would have though of that
and that I was mistaken.

But even if I have spoken up then, my experience with Python-idea is
that most of the time, people would have told me "no". They would have
told me that "nobody is going to do that".

I'm getting used to it. It took me a lot of time of "path should inherit
from strings" to finally have enough people involved that we get a
solution to the problem (which ended up being __fspath__). At the
beginning, the answer was "no, there is no problem".

So basically, now I speak up, knowing that people will say "no". But at
least I said it. Maybe in 2 years we will go back to that an implement
it, or another solution to this problem.

but I haven't yet seen good examples of real code that suffers
> from non-locked policies.  Using the nurseries example doesn't count,
> as this is something that we want to have as a builtin functionality
> in 3.8.

So in we will be able to use that in 2 years.

> 
> Locking policies can lead to more predictable user experience; OTOH
> what happens if, say, aiohttp decides to lock its policy to use uvloop
> and thus make it impossible for its users to use tokio or some other
> loop implementation?

That would be very, very good.

It would make explicit that aiohttp is:

- using a custom policy
- assuming it's the only one
- not allowing another lib to use one
- rely on it to work
- has 

Re: [Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

I like it.

First, it solves the issue for policies, and let people decide how they
want to deal with the problem (drop the lib, subclass the
policy/factory, etc).

But it also solves the problem for loops, because loops are set by the
task factory, and so you can easily check somebody is changing your loop
from you locked policy and do whatever you want.

This also solves the problem of:

- task factories
- event loop life cycle hooks

Indeed, if somebody needs those, he/she can implement a custom loop,
which can be safe guarded by the policy, which is locked.

It doesn't have the drawback of my proposal of being overly general, and
is quite simple to implement. But it does let people get creative with
the stack.




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

> 
> IMHO, it is not any framework's job to check for this.  It is a
> programmer error.  

Not clearing the memory is a programmer error either but a gc helps.

Not closing a file either but using `with` help.

Not passing the proper type is a programmer error but we have type hints.

We even have assert that we can be disabled to check that the arguments
of a function match a test when debugging.

You don't need to babysit programmers so much. 

Helping to debug is not babysitting. It's a important part of the user
experience, and it's especially needed on asyncio, one of the hardest
part of the stdlib to code with.

Not if the cost is adding new APIs.

The cost of bugs is greater than the cost of API. The current API gives
no sane way to prevent the bug. You can't expect anyone to check if the
policy / loop / task factory has changed every single time in the code
you depend on it.

> 
> I agree with Andrew, if we open this precedent, next thing we know,
> Python has to provide callbacks for any internal state changes.

It's not internal, it's a public API.

  Why not
> callbacks for when modules are imported?

Good example.

We have import hooks to run code every time a module is imported :

https://www.python.org/dev/peps/pep-0302/#specification-part-2-registering-hooks

 Etc. etc.  It leads to API
> noise.  

Every features is either useful or noise. I argue that this one is useful.

Doesn't seem to be justified in this case, since I would guess
> almost all applications only change event loop policy once, during
> startup, and never again.

Yes, but when is start up ?

E.G: I'm currently working on a large QT api on an old 2.7 code base. It
has threads to avoid blocking the UI, qt events loops of course and the
tornado events loop providing a web API. The orchestration of that very
complex app requires careful initialization for a start up time of about
7s, and we had to code some wrapper and document it as the only entry
point to make sure that if a junior breaks things the error message is
very clear.

RuntimeError('The event loop should not be set when...') is way easier
to debug than an obscure down the road error on some data structure that
is incorrectly accessed.

What I'm proposing is to make that easy to implement by just letting
anyone put a check in there.

Overriding policy, loops or tasks factories are usually down for
critical parts of the system. The errors emerging from a bug in there
are very cryptic.

Asyncio design made the choice to expose very low level things. You
literally don't have this problem in languages like JS because nobody
can change those.

Now it's here, it's a footgun, and it would be nice to provide a way to
put it in a holster.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

Le 09/06/2018 à 12:33, Andrew Svetlov a écrit :
> If we consistently apply the idea of hook for internal python structure
> modification too many things should be changed. Import
> machinery, tracemalloc, profilers/tracers, name it.
> If your code (I still don't see the real-life example) wants to check a
> policy change -- just do it.
> 
> # on initialization
> policy = asyncio.get_event_loop_policy()
> 
> # somewhere in code
> if policy is not asyncio.get_event_loop_policy():
>     raise RuntimeError("Policy was changed")

You need to repeat this check everywhere because nothing guaranty a code
hasn't change it after the check.

While a call back allow you to catch it every time.

You can set either raise, warn, monkey patch, disable features, etc.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow callables in slices

[Michel Desmoulin]

Le 09/06/2018 à 11:47, Steven D'Aprano a écrit :
> On Sat, Jun 09, 2018 at 11:17:05AM +0200, Michel Desmoulin wrote:
>> Such as that:
>>
>> def starting_when(element):
>> ...
>>
>> a_list[starting_when:]
> 
>> Is equivalent to:
> [...]
>> list(dropwhile(lambda x: not starting_when(x), a_list))
>>
> 
> That looks like a slice from an index to the end of the list. Things 
> which are similar should look similar, but things which are different 
> should NOT look similar.
> 

The semantic is [start_condition:stop_condition:step].

Here condition can be an index or something more complex.

Just like you can do dictionary[key], but key can be a complex object
with a custom __hash__ executing weird computation.

Just like you can sorted() on natural values or pass a callable as a key.

Just like you can re.replace() with a string or a function.

> What would:
> 
> alist[callable:callable:callable]
> 
> do? How about this one?
> 
> alist[7:callable:-1]

ValueError("Step cannot be used when callables are part of a slice")

However:

alist[7:callable]

Would be:

list(dropwhile(lambda x: not starting_when(x), islice(alist, 7, None)))


Example, open this files, load all lines in memory, skip the first line,
then get all the line until the first comment:

import itertools

def is_commented(line):
return lines.startwith('#')

def lines():
with open('/etc/fstab'):
lines = f.readlines()[1:]
return list(itertools.dropwhile(lines, is_commented)

Becomes:

def is_commented(line):
return lines.startwith('#')

def lines():
with open('/etc/fstab'):
return f.readlines()[1:is_commented]

It's not about how much shorter is is, but it is very nice to read.

Of course I'd prefer to have slicing on generators, since here we load
the entire file in memory. But I already suggested it several times on
python-idea and the dictator killed it. For files like fstab it's ok
since they are small.
> 
> If there are not meaningful interpretations of callables as part of 
> general slice notation, then we shouldn't use slice notation as a 
> shortcut for dropwhile.
> 
> Rather than give this syntactic support, I'd rather add a new function 
> to itertools that composes takewhile and dropwhile:

It's not syntaxic support. You can already pass callables and the
interpreter accept it fine.

But the underlying types raise TypeError because they don't know how to
use that. Just like you can pass tuples, but CPython can't use them,
while numpy can.


> 
> 
> def between(iterable, startcondition, endcondition):
> it = iter(iterable)
> return takewhile(lambda x: not endcondition(x), 
>dropwhile(lambda x: not startcondition(x), it)
>)
Or accept callables in islice.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Allow callable in slices

[Michel Desmoulin]

Given 2 callables checking when a condition arises and returning True:

def starting_when(element):
...

def ending_when(element:
...
Allow:

a_list[starting_when:]

To be equivalent to:

from itertools import dropwhile

list(dropwhile(lambda x: not starting_when(x), a_list))

And:

a_list[:ending_when]

To:

from itertools import takewhile

list(takewhile(lambda x: not ending_when(x), a_list))
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Making Path() a built in.

[Michel Desmoulin]

Creating built in dynamically is not a good idea. Tools complain, new
comers wonder where it comes from, it sets a precedent for adding more
or debating about it.

Better have the debate once here, make it official or decline it
officially, and have a clean result.

Le 08/06/2018 à 21:28, Barry a écrit :
> I think you forgot to to reply to the list.
> Barry
> 
> 
>> On 8 Jun 2018, at 13:16, Michel Desmoulin  wrote:
>>
>> Creating builtin dynamically is not a good idea. tools complains, new comers 
>> wonder where it comes from, it sets a precedent for adding more or debating 
>> about it.
>>
>> Better have the debate once here, make it official or decline it officially, 
>> and have a clean result.
>>
>>> Le 06/06/2018 à 20:05, Barry Scott a écrit :
>>> I assume the the idea is that everybody has Path available without the need 
>>> to do the import dance first.
>>>
>>> If its for personal convenience you can always do this trick, that is used 
>>> by gettext to make _ a builtin.
>>>
>>> import pathlib
>>> import builtings
>>>
>>> builtins.__dict__['Path'] = pathlib.Path
>>>
>>> Now Path *is* a builtin for the rest of the code.
>>>
>>> Barry
>>>
>>>
>>>
>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

Hi Yuri,

> 
> I actually want to propose to reduce policies API surface by
> deprecating and then removing "set_child_watcher()" methods.  Besides,
> there are many other higher priority To-Do items for asyncio in 3.8,
> like implementing Trio's nursery-like objects and cancellation scopes
> or fixing tracebacks in Tasks.
> 
> That said, the above is my "IMO".  And in your email you haven't
> actually provided clear scenarios that could be solved by adding
> "event loop hooks" to asyncio.  So I have a few questions for you:
> 
> - Do you have real-life examples of libraries that abuse policies in
> some weird ways?

It's not abuse, it's just regular use of a documented feature, but a
very potent feature with deep consequences.

> - Are those libraries popular?

aiothttp comes to mind. But the fact is, I think people explicitly avoid
using policies and custom loops because they know that they can be
swapped and you can't make it a hard requirement since they can be
swapped under your node.

> - What's the actual problem they try to solve by using policies?

e.g:
https://github.com/aio-libs/aiohttp/blob/53828229a13dc72137f430abc7a0c469678f3dd6/aiohttp/worker.py

> - What problem are you trying to solve in your code that uses policies?

I have tried to create an abstraction that creates an uvloop if it
exists, or a regular loop otherwise, or integrated it to the twisted
reactor or the trio event loop if it's loaded.

I want to abstract that from the user, so I tried to put that in a
policy. But that's dangerous since it can be changed at any time, so I
gave up on it and made it explicit. Of course, if the user misses that
in the doc (hopefully, it's an company internal code so they should be
trained), it will be a bummer to debug.

Another example is the proof of concept of nurseries we talked about on
twitter:

https://0bin.net/paste/V5KyhAg-2i5EOyoK#dzBvhdCVeFy8Q2xNcxXyqwtyQFgkxlKI3u5QG0buIcT

Yet another one, with a custom loop this time:

I want to provide a fail fast mode for code using loop.run_forever. The
goal is to make it crashes when an uncaught exception occurs instead of
just log it in the console to ease debugging in a local machine.

Of course it would be best to use run_until_complete instead but you
don't always get to choose.

So we set a task factory on the loop. But, of course, you lose
everything if something changes the loop on the fly, which my code has
no idea has happened.

Another use case is to just log that the loop / policy has changed in
debug mode. It's always something I want to know anyway, because it has
consequences on my entire program since those are pretty fundamental
components.

> - Why do you think this isn't a documentation/tutorial issue?> - Can you list 
> 2-3 clear examples where having hooks would benefit an
> average asyncio user?

The best documentation is the one you don't need to write. But if I'm
being honest, I'd like to have it despite having a good warning in the
documentation.

When you run Django's manage.py runserver, it will check your code for a
lot of common issues and raise a warning or an exception, letting you
know what to do.

With those hooks, I could check if a policy or a loop is changed, and if
some things in my code depends on a policy or a loop, I can do the same
and raise a warning or an exception.

This benefit the users because they get the info exactly in context, and
not just rely on them reading the doc, understanding it, remembering it
and applying it.

And that benefits the framework writers because that's less support, and
less bug reports to deal with.

Using asyncio is scary and mysterious enough for a lot of people, so I
want to make the experience as natural as possible. I don't want people
to have to read my doc and learn what event loops and policies are for
basic usage. It's too much. But on the other hand, I do want them to be
able to debug a problem if the loop or policy is swapped.

> 
> Thank you,
> Yury

Thank you too
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Making Path() a built in.

[Michel Desmoulin]

There are very few programs that never use any path operation.

Opening a file is such a common one we have a built-in for it with
open(), but you usually need to do some manipulation to get the file
path in the first place.

We have __file__, but the most common usage is to get the parent dir,
with os or pathlib.

Websites open static files and configurations file.

GUI open files to be processed.

Data processing open data source files.

Terminal apps often pass files as a parameters.

All those paths you may resolve, turn absolute, check against and so on.
So much that pathlib.Path is one of the things I always put in a
PYTHONSTARTUP since you need it so often.

I think Path fits the bill for being a built-in, I feel it's used more
often than any/all or zip, and maybe enumerate.

Besides, it would help to make people use it, as I regularly meet dev
that keep import os.path because of habits, tutorials, books, docs, etc.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime assertion with no overhead when not active

[Michel Desmoulin]

Maybe somebody already answered this, but what's the difference between
this and the keyword "assert", which basically can be stripped of using
"python -o" ?

Le 05/05/2018 à 10:04, Eloi Gaudry a écrit :
> 
> Hi folks,
> 
> 
> I intend to add a runtime assertion feature in python. Before
> submitting a PEP, I am sending a draft to this mailing list to
> discuss whether it would make sense (above this message). I have
> actually been using it for the last 2 years and it has proven to be
> robust and has achieved its primary goal.
> 
> 
> Briefly, the idea is to add a new assert that can be switch on/off
> depending on some variable/mechanism at runtime. The whole point of
> this assert is that it should not bring any overhead when off, i.e.
> by avoiding evaluating the expression enclosed in the runtime
> assert. It thus relies on Python grammar.
> 
> 
> Thanks for your feedback.
> 
> 
> Eloi
> 
> 
>  
> 
> 
>  
> 
> 
>  
> 
> 
> Abstract
> 
> This PEP aims at offering a runtime assert functionnality, extending the
> compiletime assert already available.
> 
> 
> Rationale
> 
> There is no runtime assert relying on python grammar available. For
> diagnostics and/or debugging reasons, it would be valuable to add such a
> feature.
> 
> A runtime assert makes sense when extra checks would be needed in a
> production environment (where non-debug builds are used).
> 
> By extending the current python grammar, it would be possible to limit
> the overhead induces by those runtime asserts when running in a non
> "assertive" mode (-ed). The idea here is to avoid evaluating the
> expression when runtime assertion is not active.
> 
> A brief example why avoiding evaluating the expression is needed to
> avoid any overhead in case the runtime assert should be ignored.
> 
> ::
> 
> runtime_assert( 999 in { i:None for i in range( 1000 ) } )
> 
> 
> Usage
> 
> ::
> 
> runtime_assert( expr )
> 
> #would result in if expr and runtime_assert_active:
> 
> print RuntimeAssertionError()
> 
> 
> Implementation details
> 
> There is already an implementation available, robust enough for
> production. The implementation is mainly grammar based, and thus the
> parser and the grammar needs to be updated:
> 
>   * Python/graminit.c
>   * Python/ast.c
>   * Python/symtable.c
>   * Python/compile.c
>   * Python/Python-ast.c
>   * Python/sysmodule.c
>   * Modules/parsermodule.c
>   * Include/Python-ast.h
>   * Include/graminit.h
>   * Include/pydebug.h
>   * Grammar/Grammar
>   * Parser/Python.asdl
>   * Lib/lib2to3/Grammar.txt
>   * Lib/compiler/ast.py
>   * Lib/compiler/pycodegen.py
>   * Lib/compiler/transformer.py
>   * Lib/symbol.py
>   * Modules/parsermodule.c
> 
> 
> References
> 
> [1]
> 
>   
> 
> PEP 306, How to Change Python's Grammar
> (http://www.python.org/dev/peps/pep-0306)
> 
> 
> Copyright
> 
> This document has been placed in the public domain.
> 
>  
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add hooks to asyncio lifecycle

[Michel Desmoulin]

After years of playing with asyncio, I'm still having a harder time
using it than any other async architecture around. There are a lot of
different reasons for it, but this mail want to address one particular one:

The event loop and policy can be tweaked at any time, by anyone.

Now, it's hard enough to have to deal, manually, with a low-level event
loop. But having it exposed that much, and it being that flexible means
any code can just do whatever it wants with it, and make a mess.

Several things in particular, comes to mind:

- Changing the event loop policy
- Changing the event loop
- Spawning a new loop
- Starting the loop
- Stopping the loop
- Closing the loop

Now, if you want to make any serious project with it, you currently have
to guard against all of those, especially if you want to have proper
cleanup code, good error message and a decent debugging experience.

I tried to do it for one year, and currently, it's very hard. You have a
lot of checks to make, redundantly in a lot of places. Some things can
only be done by providing a custom event policy/loop yourself, and, of
course, expecting (aka documenting and praying) that it's used.

For a lot of things, when it breaks, the people that haven't read the
doc in depth will have a hard time to understand the problem after the fact.

Sometimes, it's just that your code use somebody else code that is not
here to read your doc anymore. Now you have to check their code to
understand what they are doing that breaks your expectations about the
loop / policy or workflow.

Barring the creating of an entire higher level framework that everybody
will agree on using and that makes messing up way harder, we can improve
this situation by adding hooks to those events.

I hence propose to add:

- asyncio.on_change_policy(cb:Callable[[EventLoopPolicy,
EventLoopPolicy], EventLoopPolicy])

- asyncio.on_set_event_loop(cb:Callable[[EventLoop, EventLoop], EventLoop])

- asyncio.on_create_event_loop(cb:Callable[[EventLoop], EventLoop])

- EventLoop.on_start(cb:Callable[EventLoop])

- EventLoop.on_stop(cb:Awaitable[EventLoop])

- EventLoop.on_close(cb:Callable[EventLoop])

- EventLoop.on_set_debug_mode(cb:Callable[[loop]])

This would allow to implement safer, more robust and easier to debug
code. E.G:

- you can raise a warning stating that if somebody changes the event
policy, it must inherit from your custom one or deal with disabled features

- you can raise an exception on loop swap and forbid it, saying that
your small script doesn't support it yet so that it's easy to understand
the limit of your code

- you can hook on the event loop life cycle to automatically get on
board, or run clean up code, starting logging, warn that you were
supposed to start the loop yourself, etc
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make asyncio.get_event_loop a builtin

[Michel Desmoulin]

Well that doesn't matter anyway. With breakpoint() we choose the
debugger implementation, so we could do the same here.

However, this would be addressing the wrong problem the problem is
verbosity. asyncio s are already aware of that, since they are working
on providing asycio.run() in 3.7.

Now they also added asyncio.get_running_loop(), which we will soon be
used everywhere instead of get_event_loop(). That should have a shorter
alias lie asyncio.loop([check_running=True]).

Currently the shortest we can do is:

import asyncio as aio


aio.get_event_loop()


This gets old very fast. It's annoying enough that we have to deal
manually with a very low level event loop every when we go from JS/Go to
Python. It would be nice to have more helpers to do so.




Le 25/05/2018 à 02:42, Ken Hilton a écrit :
> On Tue May 22 22:08:40 (-0400), Chris Barker wrote:
>> while asyncio is in the standard library, it is not intended to be THE
> async event loop implementation
> 
> I'm surprised this is true - with dedicated syntax like async def/await,
> it's still not THE async event loop implementation? As far as I know,
> "async def" is a shorthand for
> 
> @asyncio.coroutine
> def
> 
> and "await" is short for "yield from".
> 
> Sincerely,
> Ken Hilton;
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Michel Desmoulin]

Le 10/04/2018 à 00:54, Peter O'Connor a écrit :
> Kyle, you sounded so reasonable when you were trashing
> itertools.accumulate (which I now agree is horrible).  But then you go
> and support Serhiy's madness:  "smooth_signal = [average for average in
> [0] for x in signal for average in [(1-decay)*average + decay*x]]" which
> I agree is clever, but reads more like a riddle than readable code.  
> 
> Anyway, I continue to stand by:
> 
>     (y:= f(y, x) for x in iter_x from y=initial_y)
> 
> And, if that's not offensive enough, to its extension:
> 
>     (z, y := f(z, x) -> y for x in iter_x from z=initial_z)
> 
> Which carries state "z" forward but only yields "y" at each iteration. 
> (see proposal: https://github.com/petered/peps/blob/master/pep-.rst
> )
> 
> Why am I so obsessed?  Because it will allow you to conveniently replace
> classes with more clean, concise, functional code.  People who thought
> they never needed such a construct may suddenly start finding it
> indispensable once they get used to it.  
> 
> How many times have you written something of the form?:
> 
>     class StatefulThing(object):
>     
>         def __init__(self, initial_state, param_1, param_2):
>             self._param_1= param_1 
>             self._param_2 = param_2 
>             self._state = initial_state
>     
>         def update_and_get_output(self, new_observation):  # (or just
> __call__)
>             self._state = do_some_state_update(self._state,
> new_observation, self._param_1) 
>             output = transform_state_to_output(self._state, self._param_2)
>             return output
>     
>     processor = StatefulThing(initial_state = initial_state, param_1 =
> 1, param_2 = 4)
>     processed_things = [processor.update_and_get_output(x) for x in x_gen]
>     
> I've done this many times.  Video encoding, robot controllers, neural
> networks, any iterative machine learning algorithm, and probably lots of
> things I don't know about - they all tend to have this general form.  
> 

Personally I never have to do that very often. But let's say for the
sake of the argument there is a class of problem a part of the Python
community often solves with this pattern. After all, Python is a
versatile language with a very large and diverse user base.

First, why a class would be a bad thing ? It's clear, easy to
understand, debug and extend. Besides, do_some_state_update and
transform_state_to_output may very well be methods.

Second, if you really don't want a class, use a coroutine, that's
exactly what they are for:

def stateful_thing(state, param_1, param_2, output=None):
while True:
new_observation = yield output
state = do_some_state_update(state, new_observation, param_1)
output = transform_state_to_output(state, param_2)

processor = stateful_thing(1, 1, 4)
next(processor)
processed_things = [processor.send(x) for x in x_gen]

If you have that much of a complex workflow, you really should not make
that a one-liner.

And before trying to ask for a new syntax in the language, try to solve
the problem with the existing tools.

I know, I get the frustration.

I've been trying to get slicing on generators and inline try/except on
this mailing list for years and I've been said no again and again. It's
hard. But it's also why Python stayed sane for decades.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dart like multi line strings identation

[Michel Desmoulin]

Le 02/04/2018 à 07:09, Mike Miller a écrit :
> 
> On 2018-04-01 05:36, Steven D'Aprano wrote:
>> You are right that many of the prefixes can be handled by the same code:
>>
>>  rfd rfD rFd rFD rdf rdF rDf rDF
>>  Rfd RfD RFd RFD Rdf RdF RDf RDF
>>  frd frD fRd fRD fdr fdR fDr fDR
>>  Frd FrD FRd FRD Fdr FdR FDr FDR
>>  drf drF dRf dRF dfr dfR dFr dFR
>>  Drf DrF DRf DRF Dfr DfR DFr DFR
>>  # why did we support all these combinations? who uses them?
> 
> In almost twenty years of using Python, I've not seen capital string
> prefixes in real code, ever.  Sounds like a great candidate for
> deprecation?

+1



It's not like migrating would be hard: a replace is enough to fix the
rare projects doing that. And even if they missed the warning, it's a
syntax error anyway, so you will get the error as soon as you try to run
the program, not at a later point at runtime.

What about doing a poll, then suggests a warning on 3.8, removed in 4.0 ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dart like multi line strings identation

[Michel Desmoulin]

A "d" prefix to do textwrap.dedent is something I wished for a long time.

It's like the "f" one: we already can do it, be hell is it convenient to
have a shortcut.

This is especially if, like me, you take a lot of care in the error
messages you give to the user. I write a LOT of them, very long, very
descriptive, and I have to either import textwrap or play the
concatenation game.

Having a str.dedent() method would be nice, but the d prefix has the
huge advantage to be able to dedent on parsing, and hence be more
performant.

Le 31/03/2018 à 16:50, Marius Räsener a écrit :
> Hey List,
> 
> this is my very first approach to suggest a Python improvement I'd think
> worth discussing.
> 
> At some point, maybe with Dart 2.0 or a little earlier, Dart is now
> supporting multiline strings with "proper" identation (tried, but I
> can't find the according docs at the moment. probably due to the rather
> large changes related to dart 2.0 and outdated docs.)
> 
> What I have in mind is probably best described with an Example:
> 
> print("""
>     I am a
>     multiline
>     String.
>     """)
> 
> the closing quote defines the "margin indentation" - so in this example
> all lines would get reduces by their leading 4 spaces, resulting in a
> "clean" and unintended string.
> 
> anyways, if dart or not, doesn't matter - I like the Idea and I think
> python3.x could benefit from it. If that's possible at all :)
> 
> I could also imagine that this "indentation cleanup" only is applied if
> the last quotes are on their own line? Might be too complicated though,
> I can't estimated or understand this...
> 
> thx for reading,
> Marius
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Split, slice, join and return "syntax" for str

[Michel Desmoulin]

Even if replace would be a better fit, I can see why doing those 3
operations in one row can be valuable.

But, first, they are not common enough so that it's hard to do:

spam = docs.python.org"
eggs = 'wiki.' + '.'.join(spams.split('.')[1:])

It's not that long to type, and certainly is not happening in every
single script you do.

But let's say for the sake of argument you do a lot of cmd parsing, with
a lot of split and join. Just make a helper:

def rearrange(string, sep=None, start=None, stop=None, step=None):
return sep.join(string.split(sep)[start:stop:step])

And then you can do:

spam = docs.python.org"
eggs = 'wiki.' + rearrange(spam, '.', 1)

Simple, easy, no need to change Python.



Le 04/03/2018 à 09:59, Andrés Delfino a écrit :
> Hi!
> 
> I was thinking: perhaps it would be nice to be able to quicky split a
> string, do some slicing, and then obtaining the joined string back.
> 
> Say we have the string: "docs.python.org ", and
> we want to change "docs" to "wiki". Of course, there are a ton of
> simpler ways to solve this particular need, but perhaps str could have
> something like this:
> 
> spam = "docs.python.org "
> eggs = "wiki." + spam['.'][1:]
> print(eggs) #wiki.python.org 
> 
> A quick implementation to get the idea and try it:
> 
> class Mystr(str):
>     def __getitem__(self, item):
>     if isinstance(item, str):
>     return Mystr_helper(self, item)
>     else:
>     return super().__getitem__(item)
> 
> class Mystr_helper:
>     def __init__(self, obj, sep):
>     self.obj = obj
>     self.sep = sep
>     def __getitem__(self, item):
>     return self.sep.join(self.obj.split(self.sep)[item])
> 
> What are your thoughts?
> 
> Greetings from Argentina.
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Please consider adding of functions file system operations to pathlib

[Michel Desmoulin]

+1

We already merged, os.path and glob with pathlib. Let's do all os and
shutil.

It's weird enough for beginners to even sumble upon that many ways of
doing thing for FS.

Le 20/02/2018 à 23:11, George Fischhof a écrit :
> Good day all,
> 
> as a continuation of thread "OS related file operations (copy, move,
> delete, rename...) should be placed into one module" 
> 
> https://mail.python.org/pipermail/python-ideas/2017-January/044217.html
> 
> please consider making pathlib to a central file system module with
> putting file operations (copy, move, delete, rmtree etc) into pathlib.
> 
> BR,
> George
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Complicate str methods

[Michel Desmoulin]

Le 03/02/2018 à 23:04, Franklin? Lee a écrit :
> Let s be a str. I propose to allow these existing str methods to take
> params in new forms.
> 
> s.replace(old, new):
>     Allow passing in a collection of olds.
>     Allow passing in a single argument, a mapping of olds to news.
>     Allow the olds in the mapping to be tuples of strings.
> 
> s.split(sep), s.rsplit, s.partition:
>     Allow sep to be a collection of separators.
> 
> s.startswith, s.endswith:
>     Allow argument to be a collection of strings.
> 
> s.find, s.index, s.count, x in s:
>     Similar.
>     These methods are also in `list`, which can't distinguish between
> items, subsequences, and subsets. However, `str` is already inconsistent
> with `list` here: list.M looks for an item, while str.M looks for a
> subsequence.
> 
> s.[r|l]strip:
>     Sadly, these functions already interpret their str arguments as
> collections of characters.
> 

I second that proposal. I regularly need those and feel frustrated when
it doesn't work.

I even wrote a wrapper that does exactly this :
https://github.com/Tygs/ww/blob/master/src/ww/wrappers/strings.py

But because it's pure Python, it's guaranteed to be slow. Plus you need
to install it every time you need it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 13/11/2017 à 19:57, Chris Barker a écrit :
> This has gotten to be a big thread, and it's a pretty intractable
> problem, but I think there are a few fairly small things that could be
> done to at least make it a bit easier:
> 
> 1) Add python2.exe and python3.exe files to the Windows installers -- am
> I insane or did Windows used to have that? I really think it did --
> maybe got removed when py.exe was added.

More than that. Add pythonX.X. You may want to run Python3.5 and not
Python3.6.


>   1a) alternatively, we could add a "py" executable to the standard
> linux builds, so there would be THAT one way to do it. But I think
> that's a "BAD IDEA" -- the whole "py" thing is not widely know or used,
> it's not going to show up in package install instructions for a LONG
> time, (actualy we could do both anyway)

It would work better and be cleaner, but is unlikely to happen. 1) is
simpler and have a chance to happen.



> Then "python2 -m pip install" would work everywhere (only with new
> installations, but at least with newbies, that's a bit more likely ...)
> 
>

Yeah, if 1) happens, we should promote -m for pip.

> 2) Make adding to the PATH in Windows the default. I think there are
> really three user groups:
> 
>    - newbies starting from scratch -- they want it on the PATH
> 
>    - newbies with whatever left over cruft from previous installations
> on their systems -- they want it at the FRONT of their PATH. They
> SHOULD  uninstall all the cruft, but if they don't this will still work
> with as few surprises a possible.
> 
>    - not-newbies with a previous version of python they need to continue
> using. They can uncheck the box, or use py.exe

Agreed. But if an existing python exist in the path, there should be a
warning.

> 
> 
> 3) Make --user be be automatic for pip install. Not actually the
> default, but pip could do a user install if you don't have the
> permissions for a non-user install.

Breaking compat ? Not sure people will accept.

=

Should I do a PEP with a summary of all the stuff we discussed ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

General summary so far
###


This debate has been very civil so far and quite productive, so thanks
to everyone involved.

Several issues and solution proposals have been mentioned, so I will
make a summary here.

Issue 1: running Python is a different command different OS when several
versions are installed
==

Proposal A:
---

Suffix Python executable on Windows like on Unix, so that people will
type pythonX.X if they want a specify version.

Pros: easy and discoverable.

Cons: you need a lot of stuff in the system path.

Proposal B:


Provide the "py" command on Unix, so that people will type "py -x.x"

Pros: no more issues with system path. Just add this command, and
register python installations to be listed by "py".

Cons: very few people know about "py", even on windows. This would need
lots of communication. Harder to change Mac and majors Linux distros
setup than just the next windows installer.


Issue 2: there is no one and only one obvious way to do venv / pip
===

Because pip and venv are tied to a Python version, there are numerous
ways to call them.

Just for pip, you have "pip", "py -x.x -m pip", "pythonx.x -m pip" and
"pipx".

Proposal A:
---

Decide on one form, and promote it everywhere.


Pros: simple.

Cons: long and tedious process, with a huge inertia before all 3rd party
to it. Would need a "PEP 8 for Python command". Requires "issue 1" to be
solved before.


Proposal B:


Provide a pipenv-like behavior

pipenv is a tool created by the author of "requests" that automatically
"do the right thing" by default. The first time you install a package,
it creates a virtualenv for the current directory (but don't store it in
the current directory). It will then install all packages in there.
"pipenv shell" starts a shell in the virtualenv. Dependencies are
automatically dumped in a requirement files separating dev and prod.

Pros: bypass the multiple installers issue by prompting you what version
you wish the first time it creates a virtualenv. Solve the problem of
admin rights and system path.

Cons: lots of work. Either a new tool or breaking compat with pip. If
debian didn't install pip by default, this may not help.

Issue 3: pip and virtualenv are not available everywhere
===

Proposal A:


Make a mock "pip" commands for when it's not installed. venv already
does that in some cases (see attachements), alhought it does give you
the wrong command so it should be fixed.

The gist of it: if the users try to run pip install while it's missing,
it prints a warning explaining it's not installed and the command you
can run to solve the problem plus a link to more information in the
documentation.

Pros: low friction to implement.

Cons: requires to put up a hook somewhere so that each linux distro can
input "the proper commands to use" for their system when they package
python. More a workaround than a solution.

Proposal B:


Make pip and venv part of the standard and request debian that they
provide it.

Pros: straight forward.

Cons: holy war in the making.


.


Can I have +1 and -1 from each of you on each point ?

Should I start a new thread for each of them ?


Favorite personnal combination


1 - So far, I like "providing py on unix" better, but I think it's too
hard to do. So provide suffix on windows seems more achievable.

So +1 for having pythonX.X on windows.

2 - Decide on one form, and promote it everywhere is again the simplest
IMO. I love pipenv but the strings to pull to get something like this
means it probably would not happen.

If we stick to suffix for issue 1, this mean we can promote:

pythonX.X -m pip
pythonX.X -m venv

Everywhere. In the docs. In the tutorials. In teaching classes.

This will help a lot to remove all confusions and make the Python start
up experience way easier.

3 - getting angry debian devs is probably not wise. The "mock" pip/venv
command also plays very well with the 2 other solutions.

It means we can promote everywhere:

pythonX.X -m cmd

It will work most of the time. And when it doesn't, give clear
contextual steps on





___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 12/11/2017 à 19:02, Stephan Houben a écrit :
> 
> 2017-11-12 13:20 GMT+01:00 Paul Moore <p.f.mo...@gmail.com
> <mailto:p.f.mo...@gmail.com>>:
> 
> On 12 November 2017 at 06:19, Michel Desmoulin
> <desmoulinmic...@gmail.com <mailto:desmoulinmic...@gmail.com>> wrote:
> 
> > Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> > pythonx -m venv ? Wait, it's not installed by default on debian.
> 
> Seriously? Debian don't provide venv in the standard Python install?
> That's just broken.
> 
> 
> I think that it is wrong to think of Debian's "python3" package
> as purporting to provide the "standard Python install".
> 
> This is really more properly though of as the Python *runtime*
> environment, i.e. the minimum stuff you need to run programs
> written in Python.
> 

I think it's wrong to have a different setup on different plateforms.
Python is a portable language. It's also a language you use for quick
scripts and from a lot of non professional devs.

Unifying the python experience and making sure it's easy for beginners
to get started is a very important goal IMO.

Python is actually quite good at it. But we can make it even better.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 10/11/2017 à 12:41, Oleg Broytman a écrit :
> On Fri, Nov 10, 2017 at 09:50:22AM +, Paul Moore  
> wrote:
>> The biggest reason we don't add Python to PATH, as I understand it, is
>> because we need to consider the implications of people having multiple
>> versions of Python installed.
> 
>Why not fix that the same way as on Unix -- by having versioned
> executables: python27.exe, python35.exe? Then python.exe in PATH will be
> from the most recent installed Python.
> 
>> Paul
> 
> Oleg.
> 

That or provider the "py" command on Unix. Either would work.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 10/11/2017 à 09:01, Nick Coghlan a écrit :
> On 10 November 2017 at 17:05, Michel Desmoulin
> <desmoulinmic...@gmail.com> wrote:
>>
>>> Which is why we advise getting into a virtual environment ASAP, such
>>> that the only platform specific thing folks necessarily need to learn
>>> to get started is how to get to that first working virtual
>>> environment.
>>>
>>
>> You can't start by teaching virtualenv. I tried. It doesn't work. And
>> it's a terrible prerequisit if you write docs, tutorial, teaching
>> materials, etc.
> 
> You can't have it both ways - the only way we can systematically mask
> the environmental differences between Windows, Linux and Mac OS X is
> by providing tooling that actually masks those differences, which
> means introducing that tooling becomes a prerequisite for teaching.
> 
> It doesn't completely solve the problem (as getting into and out of
> the environment is still platform specific), but it does mean that the
> ubiquitous online instructions to run "pip install package-name" and
> "python -m command" will actually work once people are inside their
> working environment.
> 
> That tooling is venv:
> 
> * it ensures you have "pip" on your PATH
> * it ensures you have "python" on your PATH
> * it ensures that you have the required permissions to install new packages
> * it ensures that any commands you install from PyPI will be also on your PATH
> 
> When we choose not to use venv, then it becomes necessary to ensure
> each of those things individually for each potential system starting
> state
> 
The way we do things is not the only way. Take JS: they don't have this
problem, you npm install the same way everywhere. You don't have
virtualenv but local node_modules. And you have transpilers to help with
the langage version differences.

Now I'm not advocating we do it the JS way. I'm just saying that you are
very keen to defend a statu quo instead of offering ideas to solve the
problem.

Besides, using venv have the same issues. It's not installed on linux by
defaut. And on windows you'll have to do py -x.x -m but on mac pythonx -m.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> On Windows, which is the only platform I can reasonably comment on,
> the killer issue is that the installer doesn't make the commands
> "python" and "pip" available by default. Just checking my PC, both go
> and rust (which I installed *ages* ago) do appear to, so they "just
> work". Java sort-of also works like that (the runtime is on PATH, but
> the compilers need to be added manually).
> 
> The biggest reason we don't add Python to PATH, as I understand it, is
> because we need to consider the implications of people having multiple
> versions of Python installed. As far as I know, no other language
> allows that. But equally, most beginners wouldn't actually *have*
> multiple versions installed. So maybe we should optimise for that case
> (only one version of Python installed). That would mean:
> 

That seems reasonable. Especially since somebody with several versions
install is more likely to know how to deal with system path issues than
a total beginner installing python for the first time.

> 1. Go back to adding Python to PATH. Because our installers don't say
> "do you want to uninstall the old version", we should probably do a
> check for a "python" command on PATH in the installer, and if there is
> one, warn the user "You already have Python installed - if you are
> upgrading you should manually uninstall the old version first,
> otherwise your old installation will remain the default". We could get
> more complex at this point, but that depends on what capabilities we
> can include in the installer - I don't know how powerful the toolset
> is.

You don't even have to do that. We can detect it and prompt : "which
python version do you want to be available by default ?", and then list
command to run the alternative versions.

> 2. Explicitly document that multiple Python interpreters on one
> machine is considered "advanced", and users with that sort of setup
> should be prepared to manage PATH themselves. I'd put that as
> something like "It is possible to install multiple versions of Python
> at once, but if you do that, you should understand the implications -
> the average user should not need to do this"
> 
> We still have to deal with the fact that basically every Unix
> environment is "advanced" in the above sense (the python2/python3
> split). I don't have a solution for that (other than "upgrade to
> Windows" ;-)).

Provide the "py" command on linux and mac. And make it the default
recommended way in the documentation.

>> It doesn't completely solve the problem (as getting into and out of
>> the environment is still platform specific), but it does mean that the
>> ubiquitous online instructions to run "pip install package-name" and
>> "python -m command" will actually work once people are inside their
>> working environment.
>>
>> That tooling is venv:
>>
>> * it ensures you have "pip" on your PATH
>> * it ensures you have "python" on your PATH
>> * it ensures that you have the required permissions to install new packages
>> * it ensures that any commands you install from PyPI will be also on your 
>> PATH
>>
>> When we choose not to use venv, then it becomes necessary to ensure
>> each of those things individually for each potential system starting
>> state
> 
> Currently, the reality is that people use virtualenv, not venv. All
> higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
> support). Enhancing the capabilities of venv is fine, but promoting
> venv over virtualenv involves technical challenges across the whole
> toolset, not just documentation/education.
> 
> But agreed, once we get people into a virtual environment (of any
> form) the portability issues become significantly reduced. The main
> outstanding issue is managing multiple environments, which could be
> handled by having a special "default" environment that is the only one
> we'd expect beginners to use/need.

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
pythonx -m venv ? Wait, it's not installed by default on debian.

And then virtualenv have their own issues:

- it's a contextual mode that you need to activate AND be aware of at
all time
- you need to config tooling to use it (IDE, builders, etc)
- the location of virtualenv is very important
- you should not commit it to git
- you can't relocate it easily
- install jupyter / mymy / flake8 outside a virtualenv and the command
will still seems to work inside a virtualenv it's not installed in. With
terrible consequences.

Virtualenvs are a hard tool to use for beginners.

A lot of people on this list have forgotten their early years it seems.

> Also on Windows, the per-user bin directory isn't added to PATH even
> if you add the system Python to PATH in the installer.
> 
>> That said, I think there is one improvement we could feasibly make,
>> which would be to introduce the notion of a "default user environment"
>> into `venv`, such that there was a single "python -m venv shell"
>> command that:
>>
>> * created a default 

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 07/11/2017 à 14:06, אלעזר a écrit :
> On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan <ncogh...@gmail.com
> <mailto:ncogh...@gmail.com>> wrote:
> 
> On 7 November 2017 at 03:52, Michel Desmoulin
> <desmoulinmic...@gmail.com <mailto:desmoulinmic...@gmail.com>> wrote:
> 
> > And assume that stuff in any tutorial you make they know this stuff.
> >
> > This is a strong barrier or entry IMO.
> 
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.
> 
> 
> I don't know if you are referring to Microsoft Windows here, but I want
> to note that from my personal experience the Windows subsystem for Linux
> ("Bash on Ubuntu on Windows") is easy to work with, so making Windows
> feel (CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy
> it is for students to set up, but it is an option at least.
> 

It doesn't solve anything:

- requires windows 10
- requires having the several Go install first
- mix windows and linux, making the whole thing confusing to beginers
- we still have the heterogenous setup between the 2 os being a problem.

I love WSL, but it's not a solution to that particular problem.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> Which is why we advise getting into a virtual environment ASAP, such
> that the only platform specific thing folks necessarily need to learn
> to get started is how to get to that first working virtual
> environment.
> 

You can't start by teaching virtualenv. I tried. It doesn't work. And
it's a terrible prerequisit if you write docs, tutorial, teaching
materials, etc.

>> - install pip manually on linux
> 
> s/Linux/Ubuntu/
> 
> Other distros (like Fedora) provide pip by default.

s/Ubuntu/Debian derivates

Debian, Ubuntu, Linux Mint, Linux Tail, etc.

> 
>> - make sure the system path is correctly set
> 
> Recent python.org Windows installers do this automatically, but there
> are unfortunately still lots of ways for things to go wrong.

Just gave a training for python 3.6 this week. It doesn't. You need to
explicitly check a box to do so. Box that no beginer would ever check
without being told to. Actually a box that some of my students even
don't check despite being emphatically told to.

> 
>> Stuff that they will forget on the next install, or miss when changing
>> plateform
> 
> Yes, because computers are awful, and incredibly user hostile. We
> don't have a magic wand to wave to fix that.

No, but we can make some adjustment to make lifes easier. For this
particular issue, checking the box by default would help beginners.
Sysadmin not wanting it would understand the problem and be able to take
the proper decision.

> 
>> And assume that stuff in any tutorial you make they know this stuff.
>>
>> This is a strong barrier or entry IMO.
> 
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.

That is  avery pessimistic that would prevent use from making any
progress in any thing. All the other participants offered at least 3
solutions to problems. They are workable. They may help. We should try them.

> 
> This is a big part of why mobile client devices with cloud backends
> are so popular, even for development purposes: they allow for a much
> simpler developer experience that avoids decades of accumulated cruft
> in the desktop operating system command line experience. Even there
> though, you're faced with the fact that once you choose a provider,
> whatever you do there will probably be locked into that provider and
> not transferable elsewhere.

I agree.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 07/11/2017 à 22:26, Paul Moore a écrit :
> On 7 November 2017 at 20:38, Chris Barker  wrote:
>> On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans  wrote:
>>>
>>> As Ivan said earlier, perhaps the Windows installers should provide a
>>> "python3" executable, so "python3 -m pip" works everywhere.
>>
>> absolutely! I really, really thought it did (I'm amazed I never heard
>> from a single student getting bit by that...)
> 
> On Windows, use py -X.Y to select the exact version of Python you
> want. Maybe Unix should have a launcher like this, too? It doesn't
> really need to be any more complex than
> 
> 
> exec pythonX.Y $@

That's my whole point. We should find a way to unify each of those issues.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> there can be multiple python2 or 3s too...
> 
> at least with:
> 
> python? -m pip install
> 
> you will get the pip that matches the python you use...

Not on windows. You have the py command. My all point is that each of
the specific issues I mentioned are A and B but not C propositions. It's
not unified.

> 
> with ensurepip, having pip no installed in a python is getting less
> common, so maybe this isn't needed anymore, but

ensurepip may depend of youself having an internet connection when you
install it. And without a proxy. And it's not used on debian flavours.

> 
> -CHB
>  
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR            (206) 526-6959    voice
> 7600 Sand Point Way NE   (206) 526-6329    fax
> Seattle, WA  98115       (206) 526-6317    main
> reception
> 
> chris.bar...@noaa.gov 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> Isn't pip installed by default on Windows and OSX? On Linux some
> distribution will probably do something unexpected and mess up your
> grand plan.

Some installers download pip, which won't work if you don't have an
internet connection or a proxy setup.

But my point is, all this should be unified. All the points I mentioned
worked in a A and B but not C, or a combination of those, different for
each point. This is very unfortunate to write docs, tutorials, or any
teaching material.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> I.e., Install under the "Python 3.6" start menu an additional
> "Python command prompt", which will
> start cmd.exe with an appropriate PATH so that python and pip
> run without further prefix.
> 
> That way, the installer still doesn't need to mess with global PATH and
> you can
> easily have multiple versions of Python, each with their own
> "Python command prompt" submenu.
> 
> At least for Windows users this would simplify the situation a bit.


They already do that on windows. But adding "yet another way" to do
things differently on different OS doesn't feel like a solution ot me.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 23:48, Chris Barker a écrit :
> On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin
> <desmoulinmic...@gmail.com <mailto:desmoulinmic...@gmail.com>> wrote:
> 
> I know and you still:
> 
> - have to use py -m on windows, python3 linux, python in virtualenv...
> 
> 
> can't you use python3 -m pip install .
> 
> everywhere?

No: on windows you don't have python3, you have py -x.x. On linux you
can't pip install, you need --users, admin rights or a virtualenv.

> 
> That's what I tell my beginner students to do, and I've never had a
> problem. (nce they got Python installed right in the first place) For that:
> 
> The python.og installers for Windows and Mac pretty much 
> "just work"

Nope. E.G: in windows, if you don't check "add python executable to
system path" people won't be able to type "python" in the shell. It's
unchecked by default.

> 
> Linux is a different story, but Linux users are more comfortable with eh
> whole idea of command lines and packages, etc -- so my Linux users have
> never been the hangup.

Not anymore. There is a wave of beginners arriving in an industry using
linux but being unfamiliar with the system. The problem is not that it's
hard. The problem is that it's work to discover it. The first Python
experience should be simple and straight forward.

Another problem is that it's not portable knowledge from an OS to another.

> 
> If/when I'm teaching data analysis for scientific computing, I go
> straight to conda, but for basic python, and most web Development,
> pyton.org <http://pyton.org> python and pip work great.

You can't teach conda first. It's not portable knowledge and would not
let students benefit from the mass of online tutorials that use pip and
virtualenv.

> 
> A also DO NOT introduce virtualenv right off the bat -- it is another
> complication that is critical to real development, but not important for
> learning python.
> 
> I have done it it the past -- it did not go well
> 
> What's too bad now is that so many docs say "pip install the_package",
> and users with multiple python installs can get burnt. (though most don't)
> 

My point exactly.

> -CHB
> 
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> chris.bar...@noaa.gov <mailto:chris.bar...@noaa.gov>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Any chance on (slowly) deprecating `eval` and `exec` as builtins?

[Michel Desmoulin]

Le 07/11/2017 à 22:39, אלעזר a écrit :
> 
> 
> בתאריך יום ג׳, 7 בנוב׳ 2017, 22:59, מאת Chris Angelico
> ‏>:
> 
> 
> -1 on hiding eval/exec; these features exist in many languages, and
> they're identically dangerous everywhere. Basically, use eval only
> with text from the owner of the system, not from anyone untrusted.
> 
> 
> I am sorry. I don't understand the reasons you are giving here. One
> sentence is a fact, and I agree with the other, so I must be missing
> something.
> 
> Elazar 
> 
>
Also: why are eval and exec even builtins ? Builtins are for stuff you
use often. I never saw in my entire career anybody use it often.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> I don't see anything particularly bogging here.
> It's always like this when you have multiple versions of the same
> software on the system. There's only one PATH, after all.
> 
> Heck, *the mere fact that Python allows to work like this is already a
> huge leap forward.* **Doing this cross-platform with exactly the same
> steps is something few could believe was even possible a couple of years
> ago!**

I agree.

> 
> To simplify things with Python, i do the following:
> * use the system's Python whenever possible

So python 2.7 on mac and some linux or none for windows...

> * if using something else, only install the one version/environment that
> I'm using day-to-day and add it to PATH (system's if safe & convenient,
> personal otherwise)
> * prefer the system's/environment's package manager to pip to install
> 3rd-party modules, too, if there is one.

We can't solve the situation perfectly, but we can unify a bit. E.G:

- provide the "py" command on all OSes to avoid the various naming and
aliases of python
- promote it in documentation
- promote the use of py -x.x -m pip instead of the myriads of alternatives
- provide an empty pip and venv module. If they are not here, py -m pip
says "your plateform doesn't provide pip by default, please do x" to
install it. With "" being plateform specific.
- check "add python executable to system path" on the windows installer
once by defaut




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 09:50, Stephan Houben a écrit :
> Hi Michel,
> 
> That's exactly why I proposed a `pip` function available from the Python
> prompt.
> I suppose you could still tell your students to copy/paste the following
> into their
> Python interpreter.
> 
> def pip(args):
>     import sys
>     import subprocess
>     subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>     print("Please restart Python now to use installed or upgraded
> packages.")


You still need to install pip on platforms it's not available. And do
that in a virtualenv or do --user. And ask for a restart.

> I suppose an alternative is to set up jupyterhub
> 
> https://jupyterhub.readthedocs.io/en/latest/
> 

I love jupyter, but you can't limit yourself to it. That won't give them
autonomy and will limit very much what they can do. E.G: You can't use
tkinter in jupyter easily.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> Software Carpentry starts out with the Anaconda distribution, as it
> not only improves the cross-platform UX consistent situation, it also
> deals with the external binary dependency problem (at least for the
> core set of packages provided either natively or via conda-forge).
> 

Yeah but the knowledge is not transposable. You can't understand a lot
of tutorials that uses pip. Plus it has licences issues. And if somebody
starts mixing conda and pip, it can results in strange things.

I do use anaconda, but you can't rely solely on it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 09:47, Nick Coghlan a écrit :
> On 6 November 2017 at 16:47, Michel Desmoulin <desmoulinmic...@gmail.com> 
> wrote:
>> I really want some people from this list to discuss here so we can find
>> a way to either unify a bit the way we install and use pip, or find a
>> way to express a tutorial that always works for people on the most
>> popular platforms and spread the word so that any doc uses it.
> 
> https://docs.python.org/3/installing/#basic-usage is as close as we've
> been able to get to that for the time being.

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...
- install pip manually on linux
- make sure the system path is correctly set

Stuff that they will forget on the next install, or miss when changing
plateform.

And assume that stuff in any tutorial you make they know this stuff.

This is a strong barrier or entry IMO.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> Do you know about "fades"?
> 
> https://fades.readthedocs.io/en/release_6_0/

So when they read any tutorial online they don't have reusable knowledge ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

> 
> 
> This is pretty much how I thought things worked for all built-in
> packages until this thread came up.  Is there any reason to not do this
> for all of stdlib?
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

Yes: https://www.python.org/dev/peps/pep-0206/#batteries-included-philosophy

I can't find it but there is a great story about a guy needing to do a
source code analysis in total isolation (not even a usb key or a cd) and
the computer has nothing installed but Python.

He gets by because Python is very useful "as is".

I can say the same.

When I log in on bare debian / centos or if I'm on a locked mac os. When
i'm on a place or in a remote place in Africa. When I'm with customers
locking down all the network with NDA, security badge and paranoia for
desert...

I can count on Python to be there for me and hashlib the stuff, etree
thingy, zipfile the foo and uuid all the bars.

It's a major feature.

I can live without typing, but I would hate having to pip install a json
parser.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Hello,

Today I'm going to give a training in Python again.

And again it will go the same way.

On Mac I will have to make people install python, then tell them to use
pip3.

On Windows, I will have to warn them about checking the "add python
executable to system path" (that one of them will ALWAYS miss anyway).
Then tell them to use py -3.x -m pip because some of them will have
several versions of Python installed.

Then on linux, I will tell them to install python-pip and python-venv
and use python3 -m pip.

I'll talk about --user, but commands won't be usable on some machine
where the Scripts or bin dir is not in the system path.

Then I will make them create a virtualenv so that they can avoid messing
with their system python and finally can just use "pip install" like in
most tutorials on the Web.

And eventually I'll talk about pipenv and conda. The first one so they
don't have to think about activating the virtualenv everytime, or pip
freeze, or create the venv, or add it to gitignore, etc. The second
because anaconda is very popular on windows.

There is no way a beginner is going to get any that by themselves
without a lot of time and pain. They will read some tutorial on the web
and struggle to make sens of what pip is and why "pip install" doesn't
work and why "python sucks".

I think Python is offering an incredible experience for first timer.
However, the whole "where is walpip" shenanigans is not one of them.

I really want some people from this list to discuss here so we can find
a way to either unify a bit the way we install and use pip, or find a
way to express a tutorial that always works for people on the most
popular platforms and spread the word so that any doc uses it.

Michel

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

Le 06/11/2017 à 07:07, Nick Coghlan a écrit :

> It's the default on Unix as well - you have to do "make install
> ENSUREPIP=no" to avoid getting it. (And some distros also modify their
> Python installations so that pip is missing by default)

On debian and derivatives (so Ubuntu) you need to install python-pip to
be able to use pip.

Now it's annoying already. Because you have to write every tutorial to
include a special case for them. But at least it's not a required step
to run your program.

However, if you do code using type hints and typing is not installed,
you can't even run the program without installing something. So we
finally have Python 3 by default on most Linux system, but still would
not be able to assume we can run a modern script on it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

I could see the typing module staying but:

- typing_extension (or the new official external module) append types in
the typing module;
- if typing_extension is not installed or too old and missing some
types, any missing type being accessed returns some kind of mock object
that avoid crashing;
- any running type checker encountering mocked types has to show a warning
- you can pip install typing (and it's mocking abilities) in python 2.

This way somebody without the 3rd party module can still use the code
without crashing it. Having types out of date or missing a brand new one
won't crash the code, only have a warning in mypy or pycharm.

This solves the compatibility problem, but also another major problem
currently: several projects I worked on had a fake typing module I could
use it in python 2 and 3 without having to manually install typing.
Sometime I also create scripts and leave type annotations, and my users
really don't want to install the exact typing extension to make it work.
And I don't want to remove the annotations manually after ward.

Le 03/11/2017 à 15:51, Chris Angelico a écrit :
> On Sat, Nov 4, 2017 at 1:49 AM, Guido van Rossum  wrote:
>> On Fri, Nov 3, 2017 at 7:45 AM, Chris Angelico  wrote:
>>>
>>> On Sat, Nov 4, 2017 at 1:35 AM, Guido van Rossum  wrote:
 [A copy from https://github.com/python/typing/issues/495 to get more
 people's attention to this issue.]

 I'm wondering if we should remove typing from the stdlib. Now's the time
 to
 think about this, as the feature freeze for 3.7 is about 12 weeks away.

 Cons:

 People have to depend on a PyPI package to use typing (but they do
 anyway
 for typing_extensions)
>>>
>>> If the lazy evaluation of annotations (PEP 563) also lands in 3.7,
>>> then this would be a very minor downside. You'd need to pip-install
>>> typing as well as mypy *for the actual type checking*, but at run
>>> time, you could ignore both (all those List[...] annotations would be
>>> stored unevaluated). Otherwise, it'd mean that any project that makes
>>> use of type hints would require typing as a run-time dependency.
>>
>>
>> This would not work if you use TypeVar, NewType, or any kind of type alias
>> involving things imported from typing (e.g. Union or TypedDict). Also
>> cast().
> 
> Ah, I forgot about those - they're not just used in the annotations.
> Oh well, was a nice idea.
> 
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Michel Desmoulin]

Given that:

- it doesn't break anything;
- the behavior makes sense;
- it can avoid the machine freezing when one is not careful;

It can be a good idea.

Those generators could have iterators that are not themselves and have a
__contains__ method behaving accordingly.

I only did the mistake of using "in" on those once, but it did happen. I
can imagine everybody do the same at least once. It's not a huge help,
but the joy of using python is a collection of small things after all.

Le 17/10/2017 à 05:27, אלעזר a écrit :
> 
> 
> בתאריך יום ג׳, 17 באוק׳ 2017, 00:13, מאת Terry Reedy ‏ >:
> 
> On 10/15/2017 9:12 PM, Jason Campbell wrote:
> ...  
> > itertools.cycle could use membership from the underlying iterable
> 
> If the underlying iterable is an iterator, this does not work.  You
> could define a Cycle class that requires that the input have
> .__contain__.
> 
> > itertools.repeat is even easier, just compare to the repeatable
> element
> 
> Again, create a Repeat(ob) class whose .__iter__ returns repeat(ob) and
> that has .__contains__(x) returning 'x == ob'.
> 
> > Does anyone else think this would be useful?
> 
> itertools is not going to switch from iterators to non-iterator
> iterables.
> 
> 
> It doesn't have to switch, and does not have to _require_ the input to
> define __contains__ method for the proposal to be meaningful. It can
> work with the same kind of iterables as its inputs, delegating whenever
> possible. I'm sure there are good reasons not to do it, but that's not
> an either/or decision.  
> 
> Elazar 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add pop_callback to deque signature

[Michel Desmoulin]

My initial proposal would have been to add a data structure to allow
something like deque, but allowing cheaper random accesses. However I
realized it was a use case more suitable for an external lib.

Thinking about how I would implement such a new data structure, I
imagined several possibilities, one of them would be to couple a deque
with a dict or a list.

However, sharing data between a deque and another data structure is hard
because, while you can easily hook on the element going in (since you
put them in yourself), there is no efficient way to get back an element
on its way out.

On lists or dicts, if you remove an element, you can pop() it and you
get back the removed element.

On deque, if you set a maxlen of 5 and add a 6th element, if you want to
get the element that has been removed, you need to check if the maxlen
has been reached, and if yes, get a reference to the first element, then
add the new one. It's inconvenient and of course slower than it needs to
be given that deque are quite fast.

So my more modest and realistic proposal would be to add a callback on
deque signature:

collections.deque(iterable, maxlen, pop_callback)

pop_callback would accept any callable, and call it everytime an element
is removed from the deque, allowing third party libraries to then do
whatever they need with it.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Move some regrtest or test.support features into unittest?

[Michel Desmoulin]

Can you elaborate on _why_ you think something is good for core/a plugin
? Cause right now it's impossible to know what's the logic behind.

Le 17/09/2017 à 22:31, Antoine Pitrou a écrit :
> On Wed, 13 Sep 2017 15:42:56 +0200
> Victor Stinner 
> wrote:
>> I would like to see if and how we can integrate/move some regrtest
>> features into the unittest module. Example of regrtest features:
>>
>> * skip a test if it allocates too much memory, command line argument
>> to specify how many memory a test is allowed to allocate (ex:
>> --memlimit=2G for 2 GB of memory)
> 
> That would be suitable for a plugin if unittest had a plugin
> architecture, but not as a core functionality IMO.
> 
>> * concept of "resource" like "network" (connect to external network
>> servers, to the Internet), "cpu" (CPU intensive tests), etc. Tests are
>> skipped by default and enabled by the -u command line option (ex: "-u
>> cpu).
> 
> Good as a core functionality IMO.
> 
>> * track memory leaks: check the reference counter, check the number of
>> allocated memory blocks, check the number of open file descriptors.
> 
> Good for a plugin IMO.
> 
>> * detect if the test spawned a thread or process and the
>> thread/process is still running at the test exit
> 
> Good for a plugin IMO.
> 
>> * --timeout: watchdog killing the test if the run time exceed the
>> timeout in seconds (use faulthandler.dump_traceback_later)
> 
> Good for a plugin IMO.
> 
>> * multiprocessing: run tests in subprocesses, in parallel
> 
> Good as a core functionality IMO.
> 
>> * redirect stdout/stderr to pipes (StringIO objects), ignore them on
>> success, or dump them to stdout/stderr on test failure
> 
> Good for a plugin IMO.
> 
>> * --slowest: top 10 of the slowest tests
> 
> Good for a plugin IMO.
> 
>> * --randomize: randomize test order
> 
> Will be tricky to mix with setupClass.
> 
>> * --match, --matchfile, -x: filter tests
> 
> Good as a core functionality IMO.
> 
>> * --forever: run the test in a loop until it fails (or is interrupted by 
>> CTRL+c)
> 
> Good for a plugin IMO.
> 
>> * --list-tests / --list-cases: list test files / test methods
> 
> Good as a core functionality IMO.
> 
>> * --fail-env-changed: mark tests as failed if a test altered the environment
> 
> Good for a plugin IMO.
> 
>> * detect if a "global variable" of the standard library was modified
>> but not restored by the test:
> 
> Good for a plugin IMO.
> 
>> * test.bisect: bisection to identify the failing method, used to track
>> memory leaks or identify a test leaking a resource (ex: create a file
>> but don't remove it)
> 
> Good as a core functionality IMO.
> 
>> I started to duplicate code in many files of Lib/test/test_*.py to
>> check if tests "leak running threads" ("dangling threads"). Example
>> from Lib/test/test_theading.py:
>>
>> class BaseTestCase(unittest.TestCase):
>> def setUp(self):
>> self._threads = test.support.threading_setup()
>>
>> def tearDown(self):
>> test.support.threading_cleanup(*self._threads)
>> test.support.reap_children()
>>
>> I would like to get this test "for free" directly from the regular
>> unittest.TestCase class, but I don't know how to extend the unittest
>> module for that?
> 
> Instead of creating tons of distinct base TestCase classes, you can just
> provide helper functions / methods calling addCleanup().
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Use __all__ for dir(module) (Was: PEP 562)

[Michel Desmoulin]

Le 18/09/2017 à 06:40, Cody Piersall a écrit :
> On Sun, Sep 17, 2017 at 9:49 PM, Guido van Rossum  wrote:
>> I ave to agree with the other committers who already spoke up.
>>
>> I'm not using tab completion much (I have a cranky old Emacs setup), but
>> isn't making tab completion better a job for editor authors (or
>> language-support-for-editor authors) rather than for the core language? What
>> editor are you using that calls dir() for tab completion?
>>
>>> From my perspective, the big benefit of this change is that
>>> tab-completion will get better for libraries which are already
>>> defining __all__.  This will make for a better REPL experience.  The
>>> only code in the stdlib that broke were tests in test_pkg which were
>>> explicitly checking the return value of dir().  Apart from that,
>>> nothing broke.
> 
> I'm sorry, I should have been more specific here. The tab completion
> provided by Jupyter uses dir() to provide the relevant tab-completion
> options.  I was motivated to put this PR together whenever someone (I
> think Nathaniel Smith) was talking about setting a custom __dir__ on a
> module by overriding class, and IIRC his motivation was so that no one
> tab-completes to use a deprecated attribute.  I spend a _lot_ of time
> in a Jupyter environment, so most of my tab completion is provided by
> whatever dir() returns.  I think this is a pretty common setup.

In that case, the problem is that jupyter should check up __all__ and
act on it. Potentially breaking the language for that seems very overkill.

I'm sure very few code actually depends on the current dir() behavior,
but it's more about the social contract of not breaking things unless
there is a very good reason.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 562

[Michel Desmoulin]

If I recall there was a proposal a few months for a "lazy" keyword that
would render anything lazy, including imports.

Instead of just adding laziness on generators, the  on imports, then who
knows where, maybe it's time to consider laziness is a hell of a good
general concept and try to generalize it ?

For imports, that would mean:

lazy from module import stuff
lazy import foo

For the rest,

bar = lazy 1 + 1

When you think about it, it's syntaxic sugar to avoid manually wrapping
everything in functions, storying stuff in closure and calling that later.

Le 12/09/2017 à 10:26, Ivan Levkivskyi a écrit :
> @Anthony
>> module.__getattr__ works pretty well for normal access, after being
>> imported by another module, but it doesn't properly trigger loading by
>> functions defined in the module's own namespace.
> 
> The idea of my PEP is to be very simple (both semantically and in terms
> of implementation). This is why I don't want to add any complex logic.
> People who will want to use __getattr__ for lazy loading still can do this
> by importing submodules.
> 
> @Nathaniel @INADA
>> The main two use cases I know of for this and PEP 549 are lazy imports
>> of submodules, and deprecating attributes.
> 
> Yes, lazy loading seems to be a popular idea :-)
> I will add the simple recipe by Inada to the PEP since it will already work.
> 
> @Cody
>> I still think the better way
>> to solve the custom dir()  would be to change the module __dir__
>> method to check if __all__ is defined and use it to generate the
>> result if it exists. This seems like a logical enhancement to me,
>> and I'm planning on writing a patch to implement this. Whether it
>> would be accepted is still an open issue though.
> 
> This seems a reasonable rule to me, I can also make this patch if
> you will not have time.
> 
> @Guido
> What do you think about the above idea?
> 
> --
> Ivan
> 
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding new lines to "Zen of Python"

[Michel Desmoulin]

Zen would suppose to remove things from it, not add.

Le 08/09/2017 à 14:47, Thomas Güttler a écrit :
> I curious if there are any plans to update the "Zen of Python".
> 
> What could be added to the "Zen of Python"?
> 
> What do you think?
> 
> Regards,
>   Thomas Güttler
> 
> 
> 
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Argparse argument deprecation

[Michel Desmoulin]

Le 09/08/2017 à 12:59, Ned Batchelder a écrit :
> OK, then on a more pragmatic note: why is it easier to write a callback
> than to write a simple if statement after the parsing?  Generating help
> is complex, and a common task that is closely tied to the syntax of the
> options, so it makes sense for argparse to do it.  Deprecation is
> neither complex, common, nor closely tied to the syntax of the options.
> 
> Another note about the proposal: calling it "deprecated" seems odd,
> since the proposal is really just a general-purpose callback.  argparse
> isn't generating the warning, your callback function would be doing it. 
> Why name it "deprecated"?  How is this different than the "action"
> keyword argument that argparse already provides?

I imagine something like:

def _(warn, forbid):
warn('This is deprecated') # for forbid to just put an error

parser.add_option(on_deprecated=deprecationCallback)

This does:

- provide an easy way to warn, or transition to forbid
- allow introspection to list the deprecated options
- deprecated options can be marked as such in the generated --help
- create a complex dynamic deprecation message, or just pass a short lambda

But indeed I'd like it to be able to do:

parser.add_option(on_deprecated=DeprecationWarning('meh'))
parser.add_option(on_deprecated=ValueError('meh'))

As a shortcut for simple use cases.

I still don't know how to make the distinction between deprecated and
removed from the introspection point of view.

All in all, I think it's an interesting proposal, but I'm not going to
fight over it. If it never happens, I can fit a bunch of "if" like you said.

> 
> --Ned.
> 
> 
> On 8/9/17 5:54 AM, Michel Desmoulin wrote:
>> Argparse is not just about parsing, it's about providing convenient
>> tooling associated with parsing.
>>
>> Otherwise you would not have automatically generated a "usage" message
>> or a "--help" command.
>>
>> Following your definition, those are not parsing. But there are here,
>> because we all end up coding them anyway.
>>
>> Le 09/08/2017 à 11:50, Ned Batchelder a écrit :
>>> On 8/9/17 3:56 AM, Tarek Ziadé wrote:
>>>> Hey,
>>>>
>>>> I don't think there's any helper to deprecate an argument in argparse
>>>>
>>>> Let's say you have a --foo option in your CLI and want to deprecate it
>>>> in the next release before you completely remove it later.
>>>>
>>>> My first though on how to do this by adding a new "deprecated" option to
>>>> https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_argument
>>>>
>>>> "deprecated" would be a callable that is called after the argument has
>>>> been parsed by argparse, 
>>>> so the developer can decide if they want to issue a deprecation warning,
>>>> use the parsed value or override it etc.
>>> I don't see why this is something that argparse has to do.  The
>>> semantics of options is handled by the rest of the program.  Why would
>>> the parser be issuing these warnings?  Let argparse parse the options,
>>> then let other code deal with what they *mean*.
>>>
>>> --Ned.
>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Argparse argument deprecation

[Michel Desmoulin]

+1, but I would make "deprecated" either a warning, an exception or a
callable.

This way to create a simple deprecation, you just provide
DeprecationWarning('This will be gone in the next release'), or
ValueError('This has been removed in 2.X, use "stuff instead"') if you
decide it's gone for good. But if you need a custom behavior, you pass
in a callable.

Le 09/08/2017 à 09:56, Tarek Ziadé a écrit :
> Hey,
> 
> I don't think there's any helper to deprecate an argument in argparse
> 
> Let's say you have a --foo option in your CLI and want to deprecate it
> in the next release before you completely remove it later.
> 
> My first though on how to do this by adding a new "deprecated" option to
> https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.add_argument
> 
> "deprecated" would be a callable that is called after the argument has
> been parsed by argparse, 
> so the developer can decide if they want to issue a deprecation warning,
> use the parsed value or override it etc.
> 
> Another interesting approach suggest by Doug Hellman, which I like as
> much, is a set of higher level options that
> provide a deprecation workflow for arguments, see
> 
> https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L441
> 
> What do you think?
> 
> Cheers
> Tarek
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Le 29/07/2017 à 18:14, Alex Walters a écrit :
> My $0.02 on the entire series of nametuple threads is… there **might**
> be value in an immutable namespace type, and a mutable namespace type,
> but namedtuple’s promise is that they can be used anywhere a tuple can
> be used.  If passing in kwargs to create the potential replacement to
> namedtuple is sensitive to dict iteration order, it really isn’t a
> viable replacement for namedtuple.

In Python 3.6, kwargs order is preserved and guaranteed. It's currently
implemented by relying on the non guaranteed dict order. But the 2 are
not linked. The spec does guaranty that for now on, kwargs order is
always preserved whether the dict order is or not.

> 
>  
> 
> I do feel like there isn’t that big of a usecase for an immutable
> namespace type as there is for a namedtuple.  I would rather namedtuple
> class creation be quicker.
> 
>  
> 
>  
> 
> *From:* Python-ideas
> [mailto:python-ideas-bounces+tritium-list=sdamon@python.org] *On
> Behalf Of *Chris Barker
> *Sent:* Friday, July 28, 2017 8:27 PM
> *To:* Ethan Furman 
> *Cc:* Python-Ideas 
> *Subject:* Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]
> 
>  
> 
> On Thu, Jul 27, 2017 at 7:42 PM, Ethan Furman  > wrote:
> 
> How I get the point[0] == 3?  The first definition of an ntuple had
> the order as x, y, and since the proposal is only comparing field
> names (not order), this (y, x) ntuple ends up being reversed to how
> it was specified.
> 
>  
> 
> I'm not sure there ever was a "proposal" per se, but:
> 
> ntuple(x=a, y=b)
> 
>  
> 
> had better be a different type than:
> 
> ntuple(y=b, x=a)
> 
> but first we need to decide if we want an easy way to make an
> namedtuple-like object or a SimpleNemaspace-like object
> 
>  
> 
> but if you are going to allow indexing by integer, then order needs to
> be part of the definition.
> 
>  
> 
> -CHB
> 
> 
> -- 
> 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
> 
> chris.bar...@noaa.gov 
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple redesign goals

[Michel Desmoulin]

Le 24/07/2017 à 13:45, Ethan Furman a écrit :
> [redirecting back to list]
> 
> On 07/24/2017 04:19 AM, Michel Desmoulin wrote:
>> Le 24/07/2017 à 13:02, Ethan Furman a écrit :
>>> On 07/23/2017 10:47 AM, Michel Desmoulin wrote:
> 
>>>> I'm not sure why everybody have such a grip on the type.
>>>
>>> If I understand the goal of "a new namedtuple" correctly, it is not to
>>> come up with yet another namedtuple type -- it is to make the existing
>>> collections.namedtuple a faster experience, and possibly add another way
>>> to create such a thing.
>>>
>>> This means that the "replacement" namedtuple MUST be backwards
>>> compatible with the existing collections.namedtuple, and keeping track
>>> of type is one of the things it does:
>>
>> Is it ? Maybe we should check that, cause we may be arguing around a
>> "nice to have" for nothing.
> 
> Um, yes, it is.  Did you not read the section you snipped? [1]
> 
>> How many people among those intereted by the proposal have a strong
>> need for the type ?
> 
> Whether there is a strong need for it is largely irrelevant; it's there
> now, it needs to stay.  If we were to remove it there would need to be a
> strong need for what we gain and that it outweighs the broken backward
> compatibility commitment that we try very hard to maintain.

You are assuming a namedtuple litteral would mean collections.namedtuple
would lose the type hability. It's not the case. The litterals can be
complement, not a remplacement.

Accelerating namedtuple can be made by rewritting it in C. The litteral
namedtuple is not necessary for that.

> 
> -- 
> ~Ethan~
> 
> [1] My apologies for the first paragraph if this is a language
> translation issue and you were talking about the backwards compatibility
> and not the type tracking.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Le 24/07/2017 à 16:12, Nick Coghlan a écrit :
> On 22 July 2017 at 01:18, Guido van Rossum  wrote:
>> Honestly I would like to declare the bare (x=1, y=0) proposal dead. Let's
>> encourage the use of objects rather than tuples (named or otherwise) for
>> most data exchanges. I know of a large codebase that uses dicts instead of
>> objects, and it's a mess. I expect the bare ntuple to encourage the same
>> chaos.

This is the people working on big code base talking.

Remember, Python is not just for Google and Dropbox.

We have thousands of user just being sysadmin, mathematicians, bankers,
analysts, that just want a quick way to make a record. They don't want
nor need a class. Dictionaries and collections.namedtuple are verbose
and so they just used regular tuples.

They don't use mypy either so having a type would be moot for them.

In many languages we have the opposite problem: people using classes as
a container for everything. It makes things very complicated with little
value.

Python actually has a good balance here. Yes, Python doesn't have
pattern matching witch makes it harder to check if a nested data
structure match the desired schema but all in all, the
bloat/expressiveness equilibrium is quite nice.

A litteral namedtuple would allow a clearer way to make a quick and
simple record.

> 
> That sounds sensible to me - given ordered keyword arguments, anything
> that bare syntax could do can be done with a new builtin instead, and
> be inherently more self-documenting as a result.
> 
> Cheers,
> Nick.
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Le 24/07/2017 à 15:31, Steven D'Aprano a écrit :
> On Sun, Jul 23, 2017 at 07:47:16PM +0200, Michel Desmoulin wrote:
> 
>> I'm not sure why everybody have such a grip on the type.
>>
>> When we use regular tuples, noone care, it's all tuples, no matter what.
> 
> Some people care.
> 
> This is one of the serious disadvantages of ordinary tuples as a 
> record/struct type. There's no way to distinguish between (let's say) 
> rectangular coordinates (1, 2) and polar coordinates (1, 2), or between 
> (name, age) and (movie_title, score). They're all just 2-tuples.


You are just using my figure of speech as a way to counter argument.
It's not a very useful thing to do.

Of course some people care, there are always a few people caring about
anything.

But you just created your manual namedtuple or a namespace and be done
with it.

Rejecting completly the literal syntax just because it doesn't improve
this use case you already had and worked but was a bit verbose is very
radical. Unless you have a very nice counter proposal that makes
everyone happy, accepting the current one doesn't take anything from you.

> 
> 
> [...]
>> The whole point of this is to make it a litteral, simple and quick to
>> use. If you make it more than it is, we already got everything to do
>> this and don't need to modify the language.
> 
> I disagree: in my opinion, the whole point is to make namedtuple faster, 
> so that Python's startup time isn't affected so badly. Creating new 
> syntax for a new type of tuple is scope-creep.

You are in the wrong thread. This thread is specifically about
namedtupels literal. Making namedtuple faster can be done in many other
ways and doesn't require a literal syntax. A literal syntax, while
making things slightly faster by nature, is essentially to make things
faster to read and write.

> 
> Even if we had that new syntax, the problem of namedtuple slowing down 
> Python startup would remain. People can't use this new syntax until they 
> have dropped support for everything before 3.7, which might take many 
> years. But a fast namedtuple will give them benfit immediately their 
> users upgrade to 3.7.

Again you are mixing the 2 things. This is why we have 2 threads: the
debate splitted.

> 
> I agree that there is a strong case to be made for a fast, built-in, 
> easy way to make record/structs without having to pre-declare them. 

Do other languages have such a thing that can be checked against types ?

> But 
> as the Zen of Python says:
> 
> Now is better than never.
> Although never is often better than *right* now.
> 

I agree. I don't thing we need to rush it. I can live without it now.  I
can live without it at all.

> Let's not rush into designing a poor record/struct builtin just because 
> we have a consensus (Raymond dissenting?) that namedtuple is too slow. 

We don't. We can solve the slowness problem without having the
namedtuple. The litteral is a convenience.

> The two issues are, not unrelated, but orthogonal. Record syntax would 
> be still useful even if namedtuple was accelerated, and faster 
> namedtuple would still be necessary even if we have record syntax.

On that we agree.

> 
> I believe that a couple of people (possibly including Guido?) are 
> already thinking about a PEP for that. If that's the case, let's wait 
> and see what they come up with.

Yes but it's about making classes less verbose if I recall. Or at least
use the class syntax. It's nice but not the same thing. Namedtuple
litterals are way more suited for scripting. You really don't want to
write a class in quick scripts, when you do exploratory programming or
data analysis on the fly.

> 
> In the meantime, lets get back to the original question here: how can we 
> make namedtuple faster?

The go to the other thread for that.

> 
> - Guido has ruled out using a metaclass as the implementation, 
>   as that makes it hard to inherit from namedtuple and another
>   class with a different metaclass.
> 
> - Backwards compatibility is a must.
> 
> - *But* maybe we can afford to bend backwards compatibility 
>   a bit. Perhaps we don't need to generate the *entire* class
>   using exec, just __new__.
> 
> - I don't think that the _source attribute itself makes
>   namedtuple slow. That might effect the memory usage of the
>   class object itself, but its just a name binding:
> 
> result._source = class_definition
> 
>   The expensive part is, I'm fairly sure, this:
> 
> exec(class_definition, namespace)
> 
> (Taken from the 3.5 collections/__init__.py.)
> 
> I asked on pythonl...@python.org whether people made us of the _source 
> attribute, and the overwhelming response was that they either didn't 
> know it existed, or if they did know, th

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

I'm not sure why everybody have such a grip on the type.

When we use regular tuples, noone care, it's all tuples, no matter what.

Well in that case, let's make all those namedtuple and be done with it.

If somebody really needs a type, this person will either used
collections.namedtuple the old way, or use a namespace or a class.

If using the type "namedtuple" is an issue because it already exist,
let's find a name for this new type that convey the meaning, like
labelledtuple or something.

The whole point of this is to make it a litteral, simple and quick to
use. If you make it more than it is, we already got everything to do
this and don't need to modify the language.


Le 23/07/2017 à 18:08, Todd a écrit :
> 
> 
> On Jul 20, 2017 1:13 AM, "David Mertz"  > wrote:
> 
> I'm concerned in the proposal about losing access to type
> information (i.e. name) in this proposal.  For example, I might
> write some code like this now:
> 
> >>> from collections import namedtuple
> >>> Car = namedtuple("Car", "cost hp weight")
> >>> Motorcycle = namedtuple("Motorcycle", "cost hp weight")
> >>> smart = Car(18_900, 89, 949)
> >>> harley = Motorcyle(18_900, 89, 949)
> >>> if smart==harley and type(smart)==type(harley):
> ... print("These are identical vehicles")
> 
> The proposal to define this as:
> 
> >>> smart = (cost=18_900, hp=89, weight=949)
> >>> harley = (cost=18_900, hp=89, weight=949)
> 
> Doesn't seem to leave any way to distinguish the objects of
> different types that happen to have the same fields.  Comparing
> `smart._fields==harley._fields` doesn't help here, nor does any type
> constructed solely from the fields.
> 
> 
> What about making a syntax to declare a type? The ones that come to mind are
> 
> name = (x=, y=)
> 
> Or
> 
> name = (x=pass, y=pass)
> 
> They may not be clear enough, though.
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Having to define objects for scripts or small projects is really adding
a burden. A namedtuple litteral strike the perferfect balance between
expressivity and concision for those cases.

Le 21/07/2017 à 17:18, Guido van Rossum a écrit :
> Honestly I would like to declare the bare (x=1, y=0) proposal dead.
> Let's encourage the use of objects rather than tuples (named or
> otherwise) for most data exchanges. I know of a large codebase that uses
> dicts instead of objects, and it's a mess. I expect the bare ntuple to
> encourage the same chaos.
> 
> -- 
> --Guido van Rossum (python.org/~guido )
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JavaScript-Style Object Creation in Python (using a constructor function instead of a class to create objects)

[Michel Desmoulin]

Le 17/05/2017 à 07:22, Guido van Rossum a écrit :
> On Tue, May 16, 2017 at 8:14 PM, Juancarlo Añez  > wrote:
> 
> What I like about attrs is:
> 
>   * The class level declaration of instance attributes
>   * That the reasonable *init*, *repr*, and *eq* are generated
> 
> OK, the former should be doable using PEP 526 (the type is stored in
> __annotations__ and the default in the class dict given to the
> metaclass), while the latter should be doable using a standard metaclass
> -- assuming we can agree on what the "reasonable" __init__, __repr__ and
> __eq__ should do.
>  
> 
> I don’t like the excessive wordiness in attrs,
> 
> Really? @attr.s is wordy? :-) I think it's deadly cute. (The only
> library I've ever seen that did something worse was "monocle" which used
> @_o.)

>>> import attr
>>> @attr.s
... class Point:
... x = attr.ib(default=42)
... y = attr.ib(default=attr.Factory(list))


Is pretty wordy compared to something like another language would do
such as:

class Point:
   int x = 42
   list y = []

Now I get that:

- Python already has a similar syntax creating class attributes.
- You can't put mutable objects here.
- attr does more since it generates dunder methods.

But having an import, a decorator and verbose calls to attr.ib does not
feel like idiomatic Python at all. Python is an elegant and expressive
language. This is none of the above. Also Python is beginner friendly.
Now OPP is already hard to teach to my students, but if I have to add
this to the mix, I will just have to tell them to copy / paste it
blindly for a long time before I can get to the point they can
understand what it does.

We should be able to find a middle ground.

First, if we have something LIKE attr, should it need an import? Basic
data structures may not require an import to work. async/await are way
better than import @asyncio.coroutine.

Secondly, while I really, really, really want this feature, I think we
should not rush it.

Some ideas have to be explored.

E.G:

Adding keywords for it ? I know adding a keyword is the worst thing one
can suggest on this list ever. But it has to be mentioned because most
other languages do it this way.

class Point:
instancevar x = 42
instancevar y = lazy [] # we add a debate about this a few months ago


Adding a special syntax for it ? ruby has something similar.

class Point:
@x = 42
@@y = list

Upgrading the class constructor? It does not address the mutablility
issue though.

class Point(x=42, y=[])

Mixing concepts?


class Point(metaclass=autoclass(x=42, y=lazy [])):
pass

@args(x=42)
@factory_args(y=list)
@autodunder()
class Point:
pass

@autoclass(
x=42
y=autoclass.lazy(list)
)
class Point:
pass


Just adding attrs, which is a workaround to a missing feature in Python,
as a boiler plate and calling it a day seems unwise.

Don't get me wrong, I like attrs, I like asyncio and I like the whole
battery included concept. But we lived without it until now, so let's
not go too fast on this.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] pathlib.Path.walk

[Michel Desmoulin]

Like os.walk, but from a Path instance.

We have Path.iterdir, but it's not recursive. Which you use either
os.scandir, or os.walk. In any case, you end up doing:

import os
import pathlib

directory = pathlib.Path(get_dir())

# do things with directory

for root, dirs, files os.walk(directory):
root = Path(root)
for f in files:
f = root / f
# do something with file

Which is not that bad, but you waste a lot of time discovering how to do
that since you look first for something like Path.walk.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding an 'errors' argument to print

[Michel Desmoulin]

Le 26/03/2017 à 10:31, Victor Stinner a écrit :
> print(msg) calls sys.stdout.write(msg): write() expects text, not bytes.

What you are saying right now is that the API is not granular enough to
just add a parameter. Not that it can't be done. It just mean we need to
expose stdout.write() encoding behavior.

> I dislike the idea of putting encoding options in print. It's too
> specific. What if tomorrow you replace print() with file.write()? Do you
> want to add errors there too?

You would have to rewrite all your calls anyway, because print() call
str() on things and accept already many parameters while file.write()
doesn't.

> 
> No, it's better to write own formatter function as shown in a previous
> email.

print(encoding) is short, easy to use, unobtrusive and will be used
ponctually.

How is that using your own formatter function better ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding an 'errors' argument to print

[Michel Desmoulin]

Le 24/03/2017 à 17:37, Victor Stinner a écrit :
> *If* we change something, I would prefer to modify sys.stdout. The
> following issue proposes to add
> sys.stdout.set_encoding(errors='replace'):
> http://bugs.python.org/issue15216
> 
> You can already set the PYTHONIOENCODING environment variable to
> ":replace" to use "replace" on sys.stdout (and sys.stderr).
> 
> Victor

This is not the same. You may want to locally apply "errors=replace" and
not the whole program.

Indeed, this can silence encoding problems. So I would probably never
set in to errors at dev time except for the few places where I know I
can explicitly silence errors.

I quite like this print(errors="replace|ignore"). This is not going to
cause any trouble, and can only help.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: Query language extension to Python (PythonQL)

[Michel Desmoulin]

Hello,

I've been following PythonQL with interest. I like the clever hack using
Python encoding. It's definitely not something I would recommend to do
for an inclusion in Python as it hijack the Python encoding method,
which prevent your from... well choosing an encoding. And requires to
have a file.

However, I find the idea great for the demonstration purpose. Like LINQ,
the strength of your tool is the integrated syntax. I myself found it
annoying to import itertools all the time. I eventually wrote a wrapper
so I could use slicing on generators, callable in slicing, etc for this
very reason.

However, I have good news.

When the debate about f-strings was on this list, the concept has been
spitted in several parts. The f-string currently implemented in Python
3.6, and an more advanced type of string interpolation: the i-string
from PEP 501 (https://www.python.org/dev/peps/pep-0501/) that is still
to be implemented.

The idea of the i-string was to allow something like this:

mycommand = sh(i"cat {filename}")
myquery = sql(i"SELECT {column} FROM {table};")
myresponse = html(i"{response.body}")

Which would then pass an object to sql/sh/html() with the string, the
placeholders and the variable context then allow it to do whatever you
want. Evaluation of the i-string would of course be lazy.

So while I don't thing PythonQL can be integrated in Python the way it
is, you may want to champion PEP 501. This way you will be able to
provide a PQL hook allowing you to do something like:

pql(i"""select (x, sum_y)
   for x in range(1,8),
   y in {stuff}
   where x % 2 == 0 and y % 2 != 0 and x > y
   group by x
   let sum_y = sum(y)
   where sum_y % 2 != 0
""")

Granted, this is not as elegant as your DSL, but that would make it
easier to adopt anywhere: repl, ipython notebook, files in Python with a
different encoding, embeded Python, alternative Python implementations
compiled Python, etc.

Plus the sooner we start with i-string, the sooner editors will
implement syntax highlighting for the popular dialects.

This would allow you to spread the popularity of your tool and maybe
change the way it's seen on this list.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] get() method for list and tuples

[Michel Desmoulin]

Le 01/03/2017 à 02:23, Ethan Furman a écrit :
> On 02/28/2017 05:18 PM, Michel Desmoulin wrote:
> 
>> I love this proposal but Guido rejected it. Fighting for it right now
>> would probably be detrimental to the current proposed feature which
>> could potentially be more easily accepted.
> 
> PEP 463 has a better chance of being accepted than this one does, for
> reasons that D'Aprano succinctly summarized.
> 
> -- 
> ~Ethan~

The debate is not even over and you are already declaring a winner.
That's not really fair. Give the idea a chance and read until the end.

D'Aprano's argument is mostly "I don't encounter IndexError really often
and when I do I have this twisted one liner to get away it".

Well, that's not really a good reason to reject things for Python
because it's a language with a very diverse user base. Some bankers,
some web dev, some geographers, some mathematicians, some students, some
3D graphists, etc. And the language value obvious, readable, predictable
code for all.

Most people on this list have a specialty, because their speciality
don't see a use for the feature doesn't mean there is not one.

So I provided on my last answer an explanation of what I would use it for.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] get() method for list and tuples

[Michel Desmoulin]

> 
> I am aware of that. I'm just saying you don't have to use try...except, 
> you can use slicing instead. Obviously you have to adapt the code since 
> you are getting a list not a single item:
> 
> # may fail, if alist is empty
> if alist[0] == "spam": ...
> 
> # cannot fail
> if alist[0:1] == ["spam"]: ...
> 
...
> 
> first_item = (alist[0:1] or ["ham"])[0]

Come on, I've been doing Python for more than a decade and never saw
anybody doing that. Even reading it in a code would make me scratch my
head for a moment with a "what is it doing that for?".

You are trying to hard to provide a counter argument here.

> 
> But honestly, in my experience the number of times I actually needed 
> something like that is tiny.
...
>
> A very robust approach **for what** ?
>
> What are you trying to do? Why are you indexing into arbitrary positions
> of a list without knowing whether or not there is something there?
>
> List are not dicts and they are used differently. It is very common, and
> useful, to want to look up a key in a dict without knowing if it exists,
> and do something with a default if it doesn't exist. This is so useful

Maybe your missions involve working with people doing properly their job.

Me, I have to deal SOAP government systems, mongodb based API built by
teenagers, geographer data set exports and FTP + CSV in marina systems
(which I happen to work on right now).

3rd party CSV, XML and JSON processing are just a hundred of lines of
try/except on indexing because they have many listings, data positions
is important and a lot of system got it wrong, giving you inconsistent
output with missing data and terrible labeling.

And because life is unfair, the data you can extract is often a mix of
heterogeneous mappings and lists / tuples. And your tool must manage the
various versions of the data format they send to you, some with
additional fields, or missing ones. Some named, other found by position.

This summer, I had to convert a data set provided by polls in africa
through an android form, generated from an XML schema, send as json
using Ajax, then stored in mongodb... to an excel spread sheet (and also
an HTML table and some JS graphs for good measure).

Needingless to say I dealt with a lot of IndexError. Grepping the
project gives me:

grep -R IndexError | wc -l
33

In contrast I have 32 KeyError (most of them to allow lazy default
value), and 3 decorators.

If special syntax exist for decorators, then surely we can spare a
.get() for lists and tuples.

Apparently IndexError is an important error because if I grep the
virtualenv of the project:

grep -R IndexError | wc -l
733

Ok, it's a pretty large project with 154 dependancies, but it's still
almost 7 IndexError by package on average. So it's not a rare use case.

I also see it regularly in my classes. Students try it because they
learned it works with dict. It makes sense.

Don't dismiss a use case because you don't have it. Python is so
versatile it's used in many diverse areas.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] get() method for list and tuples

[Michel Desmoulin]

Le 01/03/2017 à 01:02, Steven D'Aprano a écrit :
> On Tue, Feb 28, 2017 at 07:10:15PM +0100, Sven R. Kunze wrote:
> 
>> 1. advantage: it looks like dict access -> allows duck typing (oh how 
>> often I'd missed that)
> 
> Dicts and lists don't duck-type because they are very different things.
> 
> We know what ["item"]*10 does. What would {"key": "value"}*10 do?
> 
> What would list.values() iterate over?
> 
> 

The fact the API is not exactly the same doesn't prevent duck typing.
Duck typing is precesily about incomplete but good enough similar API.

For the dict and list:

- you can iterate on both
- you can index both
- you can size both

Hence I can see very well functions working with both. E.G: helper to
extract x elements or a default value:

def extract(data, *args, default="None"):
for x in args:
try:
yield data[x]
except (KeyError, ValueError):
yield default

Usage:


a, b, c = extract(scores, "foo", "bar", "doh")
x, y, z = extract(items, 2, 5, 8, default=0)

I actually have this helper function.

With list.get and tuple.get, this would become:

def extract(data, *args, default="None"):
return (data.get(x, default) for x in args)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] get() method for list and tuples

[Michel Desmoulin]

I love this proposal but Guido rejected it. Fighting for it right now
would probably be detrimental to the current proposed feature which
could potentially be more easily accepted.

At least let's make it a separate thread. I would love to restart the
debate about this one. This is one of my most wanted feature ever in Python.

Le 01/03/2017 à 01:22, Rob Cliffe a écrit :
> 
> 
> On 28/02/2017 11:54, Michel Desmoulin wrote:
>> dict.get() is a very useful method, but for lists and tuples, we need to
>> rely on try/except instead.
>>
>> Can we get list.get and tuple.get as well?
> If PEP 463 "Exception-catching expressions" were accepted and
> implemented, we wouldn't need any of them:
> val = myDict[k] except KeyError: default
> val = myList[n] except IndexError: default
> Rob Cliffe
>>
>> Also, for list, a list.setdefault like the dict.setdefault would be
>> logical.
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Another use case for the 'lazy' (aka 'delayed') keyword

[Michel Desmoulin]

Good luck

Le 28/02/2017 à 16:08, Joseph Hackman a écrit :
> Yes I do think this is precisely where a lazy or delayed feature would work 
> well. I'm working on writing up a complete PEP now, and have made some 
> progress on the things left unsolved in the other thread. I will post again 
> when the PEP is closer to complete. 
> 
> Fundamentally, anything can be done, but I don't think that altering the code 
> of others for one case is Pythonic, and though it happens, I think there is 
> value in making it less necessary. 
> 
> Cheers!
> Joseph
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] get() method for list and tuples

[Michel Desmoulin]

Le 28/02/2017 à 15:45, Steven D'Aprano a écrit :
> On Tue, Feb 28, 2017 at 12:54:26PM +0100, Michel Desmoulin wrote:
>> dict.get() is a very useful method, but for lists and tuples, we need to
>> rely on try/except instead.
> 
> No you don't. You can use slicing.
> 
> alist = [1, 2, 3]
> print(alist[99:100])  # get the item at position 99

No this gives you a list of one item or an empty list.

dict.get('key', default_value) let you get a SCALAR value, OR a default
value if it doesn't exist.

It's a very different use case.

> 
> In my experience, dict.get is very useful, but list.get only *seems* 
> useful. I've written my own version:
> 
> def get(alist, pos, default=None):
> try:
> return alist[pos]
> except IndexError:
> return default
> 
> 

Based on your rational, we would just reject dict.get as well the first
time it's implemented.

> but then struggled to find a good use for it. It seems like it ought to 
> be useful, but in practice I found that it was only covering up bugs in 
> my code. 

How so ? "get the element x or a default value if it doesn't exist" seem
at the contrary, a very robust approach.

Plus it's consistent. It's only fair to expect it to exists after you
learn about dict.get.

First places where I missed it at the top of my head was *args, sys.argv
personnaly.


If I was indexing a list outside of the range of existing
> items, that's a bug, and using get() just made it hard to fix.
> 
> 
> 
>> Can we get list.get and tuple.get as well?
>>
>> Also, for list, a list.setdefault like the dict.setdefault would be logical.
> 
> What would it do?
> 
> For example, given:
> 
> alist = []
> y = alist.setdefault(10, 'a')
> 
> what will alist equal?

Fair enough.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Another use case for the 'lazy' (aka 'delayed') keyword

[Michel Desmoulin]

Le 28/02/2017 à 13:47, Markus Meskanen a écrit :
> You don't need list comprehension, you can use a for loop.
> 
> You don't need upacking you can use indexing.
> 
> And you don't need lazy, it's just convenient.
> 
> 
> With this sole argument you can add almost anything to the language as
> long as it has at least one use case where it's better than existing
> tools. I'm not against the lazy proposal in general, but attempting to
> justify a new keyword to replace this is just silly to me:
> 
> class Config(dict):
> def lazy_get(self, key, callable=None):
> if key in self:
> return self[key]
> return callable(key)

With this sole argument you can reject await / async because we could do
it with yield from and decorator.

With this solve argument you can reject comprehension because we have
map and filter.

With this solve argument you can reject defauldict because we can code
one ourself.

I mean come on, a decorator is just:

@decorator
def decorated():

Instead of

decorated = decorator(decorated)

It's just convenience and elegance.

The sweet spot for lazy is a use case where you can't easily modify the
code to get your getter and setter. You example again makes the
assumption you can easily add such a wrapper. But if you can't, then you
have to import it and wrap the config object in every module using it.
Test all that. Ensure you don't pass the wrapper by accident to
something that would break on it, etc.

And how many time do you have to do that ?

You will write your wrapper for lists ? And tuples ?

And will you test those ? document those ? Explain to others that to get
a callable, you need to pass a callable returning a callable ?

Then again for your next project or you do a library with all the stuff
it implies ?

It's neither convenient nor elegant.

lazy is not only practical, but it's also beautiful. It reads well. It
solves a problem we all have on a regular basis.

Yes we can live without it. I mean, Python is already incredibly
convenient, of course whatever we suggest now is going to be a cherry on
top of the language cake.

And yes I got a big mouth because it's a lot of work to implement it and
it's work I'm not capable to do since I can't code in C to save my life.

Does it removes the merits from lazy ?

Wouldn't you like it to just be able to drop "lazy" in your next 3.7 code ?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Another use case for the 'lazy' (aka 'delayed') keyword

[Michel Desmoulin]

Le 28/02/2017 à 13:27, Markus Meskanen a écrit :
> Well you could just use a dict subclass here with get() that takes
> callable...
> 

As I said to Angelico:


Yes but this assumes:

- I have access to the code instantiating conf;
- all code using conf are using load_from_db as a default value;
- load_from_db exists for all code using the conf object

There is always a solution to all problems, as Python is turing complete.

You don't need list comprehension, you can use a for loop.

You don't need upacking you can use indexing.

And you don't need lazy, it's just convenient.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Another use case for the 'lazy' (aka 'delayed') keyword

[Michel Desmoulin]

Le 28/02/2017 à 13:19, Chris Angelico a écrit :
> On Tue, Feb 28, 2017 at 11:04 PM, Michel Desmoulin
> <desmoulinmic...@gmail.com> wrote:
>> Instead, I think it's a good example of were 'lazy' could help. You
>> can't get simpler than:
>>
>> conf.get('setting_name', lazy load_from_db('setting_name'))
>>
> 
> Alternatively, you could define 'conf' as a subclass of dict with a
> __missing__ method:
> 
> class Config(dict):
> def __missing__(self, key):
> self[key] = load_from_db(key)
> return self[key]
> conf = Config()
> 
> Then it becomes even simpler AND less redundant:
> 
> conf['setting_name']

Yes but this assumes:

- I have access to the code instantiating conf;
- all code using conf are using load_from_db as a default value;
- load_from_db exists for all code using the conf object

There is always a solution to all problems, as Python is turing complete.

You don't need list comprehension, you can use a for loop.

You don't need upacking you can uses indexing.

And you don't need lazy, it's just convenient.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Expose a child factory using MappingProxyType in builtins

[Michel Desmoulin]

We have the immutable frozenset for sets and and tuples for lists.

But we also have something to manipulate dict as immutable datastructures:

>>> from types import MappingProxyType as idict
>>> d = idict({'a':1, 'b':2, 'c':3})
>>> d['a'] = 4
Traceback (most recent call last):
  File "", line 1, in 
d['a'] = 4
TypeError: 'mappingproxy' object does not support item assignment

We could expose this as a built type to allow the last of the most
important data structure in Python to be easily immutable.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] get() method for list and tuples

[Michel Desmoulin]

dict.get() is a very useful method, but for lists and tuples, we need to
rely on try/except instead.

Can we get list.get and tuple.get as well?

Also, for list, a list.setdefault like the dict.setdefault would be logical.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delayed Execution via Keyword

[Michel Desmoulin]

I wrote a blog post about this, and someone asked me if it meant
allowing lazy imports to make optional imports easier.

Someting like:

lazy import foo
lazy from foo import bar

So now if I don't use the imports, the module is not loaded, which could
also significantly speed up applications starting time with a lot of
imports.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delayed Execution via Keyword

[Michel Desmoulin]

> 
> One last thing: my vote is not dropping the ":" in front of they
> keyword.
> 
> 
> I think the colon has parser problems, as I showed in some examples. 
> Plus I don't like how it looks.  But I'd much rather have `a = lazy:
> stuff` than not have the construct at all, nonetheless.
> 
> 

This was a typo on my part. I prefer to AVOID the ":" in front of the
keyword.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delayed Execution via Keyword

[Michel Desmoulin]

A great proposal, although now I would have to explain to my students
the subtle difference between:

res = (print(i * i) for i in range(x))
print('foo')
print(res)

And

res = delayed [print(i * i) for i in range(x)]
print('foo')
all(res)

They seems doing something similar, but they really don't.

Overall, I still love it.

When I read about it, I immidiatly though about how Django handles
translation in models:

- define your string in english
- mark it with ugettext_lazy and NOT ugettext
- the framework delays the translation until a request comes around with
data about the user lang

The proposed featured would solve the problem nicely.

Although I'm not clear on the result of:

def stuff(arg=delayed []):

Does this mean we create a NEW list everytime in the body function ? Or
just a new one the first time than the reference stays in arg ?

Because the first behavior would solve a problem Python had with mutable
default arguments since the begining.  But that would mean the result of
"delayed []" is a concrete thing we store in arg.

The "delayed" keyword sounds a lot like something used in async io, so I
like "lazy" much more. Not only it is shorter, but it convey the meaning
of what we are doing better.

Talking about async, we need to be clear on what those do:

a = (await|yield) lazy stuff
a = lazy (await|yield) stuff (should it even allowed ?)
a = (lazy stuff(x) for x in stuff)
a = None
with open(x) as f:
a = lazy stuff() # raise IOError
print(a)

try:
a = lazy stuff() # raise
except Exception:
pass

a = lazy f'{name}' + stuff(age) # is there a closure where we store
"name"  and 'age'?

I can see a reasonable outcome for most of this, but it must be very clear.

However, I can see several very important things we need to be taking in
consederation debugging wise.

First, if there is an exception in the lazy expression, Python must
indicate in the stack trace where this expression has been defined and
where it's evaluated.

Pdb must also be able to allow easily to step in those in a coherent manner.

Evnetually we also may need to allow this:

a = lazy stuff
if a is not lazy:
print(a)

But then lazy can't be used a var name to help with the transition.

One last thing: my vote is not dropping the ":" in front of they keyword.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Using Python for end user applications

[Michel Desmoulin]

This could change when webassembly is stable. If we manage to make a
Python => webassembly compiler, I doubt it will make Python in the
browser happen. But it certainly can make Python in NodeJS happen, and
so in Electron apps.



Le 09/02/2017 à 19:56, Nick Coghlan a écrit :
> On 7 February 2017 at 15:47, Thomas Kluyver  wrote:
>> I've been thinking for a while about Python apps using Electron (Positron?
>> ;-). It's an interesting idea from the Python side, but I struggle to come
>> up with reasons why developing an Electron+Python app would be easier than
>> developing a regular Electron app. I prefer writing Python to Javascript,
>> but you'd need quite a bit of Javascript anyway, you don't have to care
>> about browser compatibility, and there would inevitably be some extra
>> friction in using two languages.
>>
>> I'm sure there are use cases where it makes sense, like if you use Python's
>> scientific computing ecosystem. But I don't know how broad they are.
> 
> I'd say the rationale for Electron/Python apps is the same as that for
> any JS frontend/Python backend configuration - JS/CSS/HTML5 is a great
> suite of technologies for defining user interfaces, but you don't
> necessarily want to be writing all your application logic in it. (You
> certainly *can*, you just may not want to)
> 
> The trade-offs are different for client-side apps (since shipping two
> different language runtimes is kinda horrible, given neither V8 nor
> CPython is particularly lightweight), but it's not *that* different
> from the traditional Python GUI app development model of depending on
> a C/C++ toolkit like Tcl/Tk, Gtk, Qt, or wxWidgets.
> 
> It's just that the modern GUI toolkit is called V8, most of the actual
> GUI bits are written in JavaScript rather than C/C++, and the language
> independent in-process bindings got fairly dramatically worse along
> the way :)
> 
> Cheers,
> Nick.
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Reduce/fold and scan with generator expressions and comprehensions

[Michel Desmoulin]

+1, especially given that reduce is not something you use very often. 
You loop and you filter everyday, but you definitely don't need the 
cumulative result of a sequence everyday. Python already have good any, 
all, sum and string concatenation stories so most of the FP usual 
suspect are taken care of.


And remember that even when we do have something missing that we use 
often, it's not always enough to convince Guido to change the language 
for it.


E.G, we have an old and recurrent debate about adding a keyword to 
assign a temporary calculation in comprehension list so that:


[x[0].upper() for x in stuff() if x[0].upper()]

can become:


[x[0].upper() as word for x in stuff() if word]

(and many other variants)

All went to a dead end. So if you want to add the accumulate feature to 
the syntax, you better  have a VERY GOOD reason.



Le 23/10/2016 à 17:59, Steven D'Aprano a écrit :

On Sun, Oct 23, 2016 at 12:57:10PM -0200, Danilo J. S. Bellini wrote:

The idea is to let generator expressions and list/set comprehensions have a
clean syntax to access its last output. That would allow them to be an
alternative syntax to the scan higher-order function [1] (today implemented
in the itertools.accumulate function), which leads to an alternative way to
write a fold/reduce. It would be nice to have something like:


[cut suggested syntax]


instead of a reduce:


[cut four existing ways to solve the problem]

Why do we need a FIFTH way to solve this problem? What you are
describing is *exactly* the use case for a reduce or fold function. Why
add special magic syntax to make comprehensions do even more?

Not everything needs to be a one liner. It's okay to import reduce to do
a reduce. Its okay to write an actual for-loop.


Actually, I already wrote a solution for something similar to that:
PyScanPrev [2].


Ah, that makes SIX existing solutions. Do we need a seventh?




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Civility on this mailing list

[Michel Desmoulin]

+1.

I read many disagreements, and people being rude and unprofessional on 
occasions, but nothing that would make me have a bad day, even when I 
was the target of it.


I feel like people are really getting hyper sensitive about 
communications. While I do prefer talking to calm rational people with a 
friendly tone, I acknowledge this is not always the case and it's ok if 
somebody go overboard from time to time.


We are not living in a perfect world, and spending a lot of effort 
trying to smooth everything out seems overkill to me.



Le 18/10/2016 à 01:10, Rene Nejsum a écrit :

Dear Brett/

I have been reading the python-idea archive from time to time over the
past years and I joined the list about a month ago to promote my “crazy”
async object idea. I did fear the response to a newcomer with an
unlikely idea, but I must say the *everyone* has been extremely nice,
writing often long answer to discussions and trying to understand where
I’m coming from with this idea. And it definitely made me try to think a
little extra before sending responses …

I did also raise an eye-brow when reading some of the comments in the
thread you mentioned, they seam a little out of touch with my experience
on other threads here.

Hope some time off will do you good, my best advice to you and others is
something that have helped me, in similar situations is the old saying
 “Other peoples opinion of you, are none of your business” :-) It took
me some years to get it, but now it helps me every time i get worked up
about something another person says to me or about me.

best
/Rene




On 17 Oct 2016, at 20:29, Brett Cannon > wrote:


Based on some emails I read in the " unpacking generalisations for
list comprehension", I feel like I need to address this entire list
about its general behaviour.

If you don't follow me on Twitter you may not be aware that I am
taking the entire month of October off from volunteering any personal
time on Python for my personal well-being (this reply is being done on
work time for instance). This stems from my wife pointing out that I
had been rather stressed in July and August outside of work in
relation to my Python volunteering (having your weekends ruined is
never fun). That stress stemmed primarily from two rather bad
interactions I had to contend with on the issue track in July and
August ... and this mailing list.

When I have talked to people about this mailing list it's often
referred to by others as the "wild west" of Python development
discussions (if you're not familiar with US culture, that turn of
phrase basically means "anything goes"). To me that is not a
compliment. When I created this list with Titus the goal was to
provide a safe place where people could bring up ideas for Python
where people could quickly provide basic feedback so people could know
whether there was any chance that python-dev would consider the
proposal. This was meant to be a win for proposers by not feeling like
they were wasting python-dev's time and a win for python-dev by
keeping that list focused on the development of Python and not
fielding every idea that people want to propose.

And while this list has definitely helped with the cognitive load on
python-dev, it has not always provided a safe place for people to
express ideas. I have seen people completely dismiss people's
expertise and opinion. There has been name calling and yelling at
people (which is always unnecessary). There have been threads that
have completely derailed itself and gone entirely off-topic. IOW I
would not hold this mailing list up as an example of the general
discourse that I experience elsewhere within the community.

Now I realize that we are all human beings coming from different
cultural backgrounds and lives. We all have bad days and may not take
the time to stop and think about what we are typing before sending it,
leading to emails that are worded in a way that can be hurtful to
others. It's also easy to forget that various cultures views things
differently and so that can lead to people "reading between the lines"
a lot and picking up things that were never intended. There are 1,031
people on this mailing list from around the world and it's easy to
forget that e.g. Canadian humour may not translate well to Ukrainian
culture (or something). What this means is it's okay to *nicely* say
that something bothered you, but also try to give people the benefit
of the doubt as you don't know what their day had been like before
they wrote that email (I personally don't like the "just mute the
thread" approach to dealing with bad actors when the muting is silent
as that doesn't help new people who join this mailing list and the
first email they see is someone being rude that everyone else didn't
see because they muted the thread days ago).

As for the off-topic threads, please remember there are 1,031 people
on this mailing list (this doesn't count people reading through gmane
or 

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Michel Desmoulin]

Regarding all those examples:

Le 14/10/2016 à 00:08, אלעזר a écrit :

Trying to restate the proposal, somewhat more formal following Random832
and Paul's suggestion.

I only speak about the single star.
---

*The suggested change of syntax:*

comprehension ::=  starred_expression comp_for

*Semantics:*

(In the following, f(x) must always evaluate to an iterable)

1. List comprehension:

result = [*f(x) for x in iterable if cond]

Translates to

result = []
for x in iterable:
if cond:
result.extend(f(x))

2. Set comprehension:

result = {*f(x) for x in iterable if cond}

Translates to

result = set()
for x in iterable:
if cond:
result.update(f(x))


Please note that we already have a way to do those. E.G:

   result = [*f(x) for x in iterable if cond]

can currently been expressed as:

   >>> iterable = range(10)
   >>> f = lambda x: [x] * x
   >>> [y for x in iterable if x % 2 == 0 for y in f(x)]
   [2, 2, 4, 4, 4, 4, 6, 6, 6, 6, 6, 6, 8, 8, 8, 8, 8, 8, 8, 8]


Now I do like the new extension syntax. I find it more natural, and more 
readable:


   >>> [*f(x) for x in iterable if x % 2 == 0]

But it's not a missing feature, it's really just a (rather nice) 
syntaxic improvement.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] async objects

[Michel Desmoulin]

On the other hand, await / async is a fantastic interface to unify all 
concurrent paradigms and asyncio already provide a bridge with threads 
and subprocess. So it kinda make sense.


Le 04/10/2016 à 18:40, Sven R. Kunze a écrit :

On 04.10.2016 13:30, Nick Coghlan wrote:

What it *doesn't* do, and what you need greenlet for, is making that
common interface look like you're using plain old synchronous C
threads.

If folks really want to do that, that's fine - they just need to add
gevent/greenlet as a dependency, just as the folks that don't like the
visibly object-oriented nature of the default unittest and logging
APIs will often opt for third party alternative APIs that share some
of the same underlying infrastructure.


Maybe, this is all a big misunderstanding.

asyncio is incompatible with regular execution flow and it's **always
blocking**. However, asyncio is perceived by some of us (including me)
as a shiny alternative to processes and threads but really isn't. I
remember doing this survey on python-ideas (results here:
https://srkunze.blogspot.de/2016/02/concurrency-in-python.html) but I
get the feeling that we still miss something.

My impression is that asyncio shall be used for something completely
different than dropping off things into a background worker. But looking
at the cooking example given by Steve Dower (cf. blog post), at other
explanations, at examples in the PEPs, it just seems to me that his
analogy could have been made with threads and processes as well.

At its core (the ASYNC part), asyncio is quite similar to threads and
processes. But its IO-part seem to drive some (design) decisions that
don't go well with the existing mental model of many developers. Even
PEP-reviewers are fooled by simple asyncio examples. Why? Because they
forget to spawn an eventloop. "async def and await" are just useless
without an eventloop. And maybe that's what's people frustration is
about. They want the ASYNC part without worrying about the IO part.

Furthermore, adding 2 (TWO) new keywords to a language has such an
immense impact. Especially when those people are told "the barrier for
new keywords is quite high!!". So, these new keywords must mean something.


I think what would help here are concrete answers to:

0) Is asyncio a niche feature only be used for better IO?
1) What is the right way of integrating asyncio into existing code?
2) How do we intend to solve the DRY-principle issue?

If the answer is "don't use asyncio", that's a fine result but honestly
I think it would be just insane to assume that we got all these
features, all this work and all those duplicated functions all for
nothing. I can't believe that. So, I am still looking for a reasonable
use-case of asyncio in our environment.

Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Clear screen command for the REPL

[Michel Desmoulin]

+1 for this. I regularly miss this feature.

Le 17/09/2016 à 13:12, João Matos a écrit :
> Hello,
> 
> In other interpreted programming languages the clear screen command
> (whatever it is) also does not clear the session.
> It just clears the screen clutter.
> 
> As I said, this would be very useful for newbies, which don't know
> anything about usercustomize or sitecustomize.
> 
> 
> Best regards,
> 
> JM
> 
> 
> On 17-09-2016 12:07, Chris Angelico wrote:
>> On Sat, Sep 17, 2016 at 8:51 PM, João Matos  wrote:
>>> I would like to suggest adding a clear command (not function) to Python.
>>> It's simple purpose would be to clear the REPL screen, leaving the >>>
>>> prompt at the top left of the screen.
>>>
>>> This is something very basic but also very useful for newbies learning
>>> Python from the REPL.
>>> After some trial and errors it is best to start with a clean screen.
>>> Clearing the screen helps clear your mind.
>>>
>> I'm not sure that it _is_ helpful, given that you're starting with a
>> clean screen but not restarting the session (so you'll still have all
>> the state from your previous work). If you want a completely fresh
>> start, just exit Python, clear the screen with a shell command, and
>> re-enter.
>>
>> The easiest way to play around with this would be to create a pure
>> Python clear() function in your usercustomize or sitecustomize, and
>> then try it in your own workflow - see whether it annoys you that it
>> doesn't change the interpreter state. Maybe it will, maybe it won't.
>>
>> ChrisA
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Null coalescing operator

[Michel Desmoulin]

I doubt very much it will be used for very complexe cases. Just like
comprehensions or ternary expressions, they are a good fit for some
specific use cases, and people will quickly catch on which one.

You rarely see nested comprehensions or ternary expressions while it's
possible to do so, because the Python communality values tend to limit
abuses.

It will be the same for this.

You won't see very complex usages, mostly things like:

val = foo[-1] except IndexError: "bar"
doh = val.attr.other except AttributeError: "default"

Those would already be coded with something similar (or maybe a chain of
getattr()) or next(iter()). It's not a huge risk.

But it's a huge convenience.


Le 13/09/2016 à 13:37, Nick Coghlan a écrit :
> On 13 September 2016 at 21:15, Rob Cliffe  wrote:
>> On 13/09/2016 04:43, Guido van Rossum wrote:
>>> Yeah, that's exactly my point. PEP 463 gives you a shorter way to
>>> catch an exception, so it gives you less motivation to find a way to
>>> write your code (or define your API) that doesn't involve catching
>>> exceptions. But APIs involving exceptions are often inferior to APIs
>>> that don't require exception catching. (Yes, I am aware of __next__()
>>> raising StopIteration -- but that API design usually doesn't require
>>> you to catch it.)
>>>
>> You surprise me.  I thought LBYL and EAFP were both approved Python idioms,
>> in some cases one being better, in some cases another, choice to be made on
>> the merits of each case (or the author's preference).  I certainly use both
>> (and sometimes time both to see which is faster).
>> Now it sounds as if you're trying to impose a style guide on the world by
>> discouraging the EAFP.  And wasn't the discussion general, not about APIs
>> specifically?
> 
> Which is preferable depends greatly on context of use, which is why
> you'll find a lot of Python APIs offer both forms - it's not *just* a
> matter of inheriting the exceptionless version from C, and then later
> adding a Python version that gives an exception instead of None or a
> user-supplied default value.
> 
> It's similar to why IEEE754 defines both quiet NaN *and* signalling
> NaN - which one you want depends on what you're doing.
> 
> In web servers, for example, you'll often have lots of fields where
> "not present" is a perfectly acceptable return value. For those, APIs
> that just return None for unknown entries are very handy, which is why
> SQL Alchemy offers both ".first()" and "one()", which mainly differ in
> how and when they throw an exception, rather than what they do when
> they succeed.
> 
> However, blindly catching *all* exceptions from a complex
> subexpression is rarely the right thing to do, so APIs that only offer
> "this may throw exceptions during normal operation under these
> circumstances" without a convenience wrapper that does the exception
> handling for you can end up being a pain to work with.
> 
> PEP 463 makes those APIs less painful to deal with, but at the cost of
> encouraging overly broad exception handlers. By contrast, fixing APIs
> on a case-by-case basis puts the design burden where it can do the
> most good: on the API designer, who can make the scope of the
> exception handling suitably narrow *inside* the API implementation,
> rather than being limited to wrapping the entire API call in
> try/except.
> 
> Cheers,
> Nick.
> 
> P.S. There are also some use cases where Look-Before-You-Leap is
> inherently subject to race conditions, and for those, exceptions are
> the only reliable signaling mechanism.
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/