Re: [Python-ideas] Should Python have user-defined constants?

[אלעזר]

בתאריך יום ג׳, 21 בנוב׳ 2017, 19:36, מאת Chris Barker ‏<
chris.bar...@noaa.gov>:

> ...
>

And what's the use-case, really? beyond the use case for all sorts of
> static typing...
>

I don't understand the question. The use case was explained before - people
want to have better ways to reason about their programs. Statically. Why
dismiss it as a non-usecase? It's helpful for both tools and humans.

It was mentioned that there are related conventions in Python. Good point.
But they are not as strong as the "convention" that tuples and strings
cannot change, and this is useful in reasoning about code - in
understanding the program, essentially.

When I read "final int x = 5;" in Java, I don't have to think about it
anymore - it's 5. When I read "X = 5" in Python, it might be a constant,
but it might also be a misnomer, or something that used to be a constant,
or a class reassigned. I still have to watch whether it will be changed,
intentionally or accidentally, by this or other module. It does not even
communicate intention in an unambiguous way.

In my opinion, the inability to control mutation in Python is the biggest
readability issue of the language (thank god monkey-patching is
discouraged).

I see why the proposal can be rejected, but I don't understand the "you
don't gain anything" responses. There are significant gains there.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[אלעזר]

On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan  wrote:

> On 7 November 2017 at 03:52, Michel Desmoulin 
> wrote:

> And assume that stuff in any tutorial you make they know this stuff.
> >
> > This is a strong barrier or entry IMO.
>
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.
>
>
I don't know if you are referring to Microsoft Windows here, but I want to
note that from my personal experience the Windows subsystem for Linux
("Bash on Ubuntu on Windows") is easy to work with, so making Windows feel
(CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy it is for
students to set up, but it is an option at least.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Any chance on (slowly) deprecating `eval` and `exec` as builtins?

[אלעזר]

Hi,

The dangers of eval and exec are obvious and well known to advanced users,
but the availability as built-in functions makes it too tempting for
beginners or even medium-level programmers. You can see questions about
these function pretty often in stackoverflow (roughly once a day
, though
sometimes the uses are legitimate).

Maybe we could start a ten-year process of deprecating the use of
`builtins.eval` (in the docs, and then with warnings)? `builtins.eval` will
be a wrapper to the real evaluation function, moved to `unsafe.eval` or
something obvious like that, so all you need to do to port your code is to
add `from unsafe import unsafe_eval as eval, unsafe_exec as exec` at the
top of the file; it will be a nice warning to the reader.

The fact that it is a wrapper will slightly slow it down and make the stack
traces noisier - both are good things, IMO.

Also, it is unfortunate that `ast.literal_eval` is less accessible than
`builtins.eval`. Giving it an alias in builtins might make it easier for
programmers (and less scary - "ast" might sound like I need a PhD to use
it).

What do you think?

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python-ideas Digest, Vol 131, Issue 106

[אלעזר]

On Tue, Oct 31, 2017 at 12:18 PM Koos Zevenhoven  wrote:

> On Tue, Oct 31, 2017 at 11:24 AM, Petr Viktorin  wrote:
>
>> On 10/31/2017 09:54 AM, Koos Zevenhoven wrote:
>>>
>>>
>>> ​I wonder if that's more easily understood if you write it along these
>>> line(s):
>>>
>>>(the_bob,) = ​(name for name in ('bob','fred') if name=='bob')
>>>
>>
>> There are (unfortunately) several ways to do it. I prefer one that avoids
>> a trailing comma:
>>
>> [the_bob] = ​(name for name in ('bob','fred') if name=='bob')
>>
>>
> ​Maybe it's just me, but somehow that list-like syntax as an assignment
> target feels wrong in somewhat the same way that (1, 2).append(3) does.​
>
>
Off topic: why can't we simply allow something like this:

(the_bob) = (name for name in ('bob','fred') if name=='bob')

Why does Python treat the parenthesis at the LHS as grouping parens?
operators are not allowed anyway; (a + (b + c)) = [1] is syntax error.

Currently

(x) = 1

works, but I can't see why should it.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[אלעזר]

בתאריך יום ג׳, 17 באוק׳ 2017, 00:13, מאת Terry Reedy ‏:

> On 10/15/2017 9:12 PM, Jason Campbell wrote:
> ...
> > itertools.cycle could use membership from the underlying iterable
>
> If the underlying iterable is an iterator, this does not work.  You
> could define a Cycle class that requires that the input have .__contain__.
>
> > itertools.repeat is even easier, just compare to the repeatable element
>
> Again, create a Repeat(ob) class whose .__iter__ returns repeat(ob) and
> that has .__contains__(x) returning 'x == ob'.
>
> > Does anyone else think this would be useful?
>
> itertools is not going to switch from iterators to non-iterator iterables.
>

It doesn't have to switch, and does not have to _require_ the input to
define __contains__ method for the proposal to be meaningful. It can work
with the same kind of iterables as its inputs, delegating whenever
possible. I'm sure there are good reasons not to do it, but that's not an
either/or decision.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 563: Postponed Evaluation of Annotations, first draft

[אלעזר]

On Mon, Sep 11, 2017 at 10:07 PM Lukasz Langa  wrote:

> This is off topic for discussion of this PEP.
>
> It would require another one (essentially an extension of PEP 484) to get
> passed for your idea to be standardized.
>

I'm not sure whether this is directed to me; so just to make it clear, I
did not propose anything here.
I will be happy for PEP 563 to be accepted as is, although the implications
of the ability to customize Python's scoping/binding rules are worth
mentioning in the PEP, regardless of whether these are considered a pro or
a con.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 563: Postponed Evaluation of Annotations, first draft

[אלעזר]

On Mon, Sep 11, 2017 at 8:58 PM Steven D'Aprano <st...@pearwood.info> wrote:

> On Mon, Sep 11, 2017 at 04:06:14PM +0000, אלעזר wrote:
> > I like it. For previous discussion of this idea see here:
> >
> https://mail.python.org/pipermail/python-ideas/2016-September/042527.html
> >
> > I don't see this mentioned in the PEP, but it will also allow (easy)
> > description of contracts and dependent types.
>
> How? You may be using a different meaning to the word "contract" than I'm

familiar with. I'm thinking about Design By Contract, where the
> contracts are typically much more powerful than mere type checks, e.g. a
> contract might state that the argument is float between 0 and 1,


def f(x: float and (0 <= x <= 1)) -> float: ...


> or that the return result is datetime object in the future. There are (at
> least?) three types of contracts: preconditions, which specify the
> arguments,


Exemplified above


> postconditions, which specify the return result,


def f(x: int, y: int) -> ret < y: # ret being an (ugly) convention,
unknown to python
...


> and invariants, which specify what doesn't change.
>

class A:
x: x != 0
y: y > x
def foo(self): ...

Of course I'm not claiming my specific examples are useful or readable. I'm
also not claiming anything about the ability to check or enforce it; it's
merely about making python more friendly to 3rd party tool support.

You didn't ask about dependent types, but an example is in order:

def zip(*x: Tuple[List[T], _n]) -> List[Tuple[T, _n]]: ... # _n
being a binding occurrence, again not something the interpreter should know

Basically dependent types, like other types, are just a restricted form of
contracts.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 563: Postponed Evaluation of Annotations, first draft

[אלעזר]

I like it. For previous discussion of this idea see here:
https://mail.python.org/pipermail/python-ideas/2016-September/042527.html

I don't see this mentioned in the PEP, but it will also allow (easy)
description of contracts and dependent types.

Elazar

On Mon, Sep 11, 2017 at 6:59 PM Lukasz Langa  wrote:

> PEP: 563
> Title: Postponed Evaluation of Annotations
> Version: $Revision$
> Last-Modified: $Date$
> Author: Łukasz Langa 
> Discussions-To: Python-Dev 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 8-Sep-2017
> Python-Version: 3.7
> Post-History:
> Resolution:
>
>
> Abstract
> 
>
> PEP 3107 introduced syntax for function annotations, but the semantics
> were deliberately left undefined.  PEP 484 introduced a standard meaning
> to annotations: type hints.  PEP 526 defined variable annotations,
> explicitly tying them with the type hinting use case.
>
> This PEP proposes changing function annotations and variable annotations
> so that they are no longer evaluated at function definition time.
> Instead, they are preserved in ``__annotations__`` in string form.
>
> This change is going to be introduced gradually, starting with a new
> ``__future__`` import in Python 3.7.
>
>
> Rationale and Goals
> ===
>
> PEP 3107 added support for arbitrary annotations on parts of a function
> definition.  Just like default values, annotations are evaluated at
> function definition time.  This creates a number of issues for the type
> hinting use case:
>
> * forward references: when a type hint contains names that have not been
>   defined yet, that definition needs to be expressed as a string
>   literal;
>
> * type hints are executed at module import time, which is not
>   computationally free.
>
> Postponing the evaluation of annotations solves both problems.
>
> Non-goals
> -
>
> Just like in PEP 484 and PEP 526, it should be emphasized that **Python
> will remain a dynamically typed language, and the authors have no desire
> to ever make type hints mandatory, even by convention.**
>
> Annotations are still available for arbitrary use besides type checking.
> Using ``@typing.no_type_hints`` in this case is recommended to
> disambiguate the use case.
>
>
> Implementation
> ==
>
> In a future version of Python, function and variable annotations will no
> longer be evaluated at definition time.  Instead, a string form will be
> preserved in the respective ``__annotations__`` dictionary.  Static type
> checkers will see no difference in behavior, whereas tools using
> annotations at runtime will have to perform postponed evaluation.
>
> If an annotation was already a string, this string is preserved
> verbatim.  In other cases, the string form is obtained from the AST
> during the compilation step, which means that the string form preserved
> might not preserve the exact formatting of the source.
>
> Annotations need to be syntactically valid Python expressions, also when
> passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
> Annotations can only use names present in the module scope as postponed
> evaluation using local names is not reliable.
>
> Note that as per PEP 526, local variable annotations are not evaluated
> at all since they are not accessible outside of the function's closure.
>
> Enabling the future behavior in Python 3.7
> --
>
> The functionality described above can be enabled starting from Python
> 3.7 using the following special import::
>
> from __future__ import annotations
>
>
> Resolving Type Hints at Runtime
> ===
>
> To resolve an annotation at runtime from its string form to the result
> of the enclosed expression, user code needs to evaluate the string.
>
> For code that uses type hints, the ``typing.get_type_hints()`` function
> correctly evaluates expressions back from its string form.  Note that
> all valid code currently using ``__annotations__`` should already be
> doing that since a type annotation can be expressed as a string literal.
>
> For code which uses annotations for other purposes, a regular
> ``eval(ann, globals, locals)`` call is enough to resolve the
> annotation.  The trick here is to get the correct value for globals.
> Fortunately, in the case of functions, they hold a reference to globals
> in an attribute called ``__globals__``.  To get the correct module-level
> context to resolve class variables, use::
>
> cls_globals = sys.modules[SomeClass.__module__].__dict__
>
> Runtime annotation resolution and class decorators
> --
>
> Metaclasses and class decorators that need to resolve annotations for
> the current class will fail for annotations that use the name of the
> current class.  Example::
>
> def class_decorator(cls):
> annotations = get_type_hints(cls)  # raises NameError on 'C'
>   

[Python-ideas] Add a more disciplined ways of registering ABCs

[אלעזר]

Hi all,

tl;dr: I propose adding a `register()` decorator, to be used like this:

@abc.register(Abc1, Abc2)
class D:
...

For preexisting classes I propose adding a magic static variable
`__registered__`, to be handled by ABCMeta:

class Abc1(metaclass=ABCMeta):
__registered__ = [D]

Both forms has the effects of calling ABCMeta.register with the
corresponding parameters.

Explanation:

It is possible to register an abstract base class B for preexisting classes
A using B.register(A). This form is very flexible - probably too flexible,
since the operation changes the type of A, and doing it dynamically makes
it difficult for type checker (and human beings) to analyze.

But what about new classes? Registering a new class is possible using
B.register as a decorator, since it already its argument:

@Iterable.register
@Sequence.register
class A: pass

However, due to restrictions on the expressions allowed inside a decorator,
this would not work with generic ABCs, which are common and heavily used in
type-checked projects:

>>> @Iterable[str].register
  File "", line 1
@Iterable[str].register
^
SyntaxError: invalid syntax

While it might be possible to infer the argument, it is not always the
case, and adds additional unnecessary burden to type-checker and to the
reader; discouraging its use. This kind of restrictions may also prevent
some other conceivable forms, such as `@type(X).register`.

Additionally, `abc.register()` is also easier to analyze as a "syntactic
construct" from the point of view the type checker, similar to the way
checkers handle some other constructs such as `enum()`, `cast()` and others.

Finally, there's uninformative repetition of `register` for multiple ABCs,
and waste of vertical screen space.

Q: Why not subclass B?
A: Since it forces the metaclass of A to be (a subclass of) ABCMeta, which
can cause inconsistency in metaclass structure for derived classes. This
issue can surface when using Enum (whose metaclass is EnumMeta):

>>> class A(Iterable, Enum): pass
...
Traceback (most recent call last):
  File "", line 1, in 
TypeError: metaclass conflict: the metaclass of a derived class must be
a (non-strict) subclass of the metaclasses of all its bases

This causes problems in typeshed, since it uses inheritance to register
`str` as a `Sequence[str]`, for example. As a result, it affects users of
mypy trying to define `class A(str, Enum): ...`

Q: Why not add to the typing module?
A: Because registering an abstract base class has runtime meaning, such as
the result of `isinstance` checks.

Q: Why not add to the standard library? why not some 3rd party module?
A:
1. Because dynamic usage should be discouraged, similarly to
monkey-patching. Note that, like monkey-patching, this has runtime
implications which cause "actions from distance".
2. Because it should be used by typeshed and mypy.

The implementation of register() is straightforward:

def register(*abcs):
def inner(cls):
for b in abcs:
b.register(cls)
return cls
return inner


Thanks,
Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[אלעזר]

On Fri, Jul 28, 2017 at 12:51 AM Chris Angelico  wrote:

> On Fri, Jul 28, 2017 at 7:22 AM, Pavol Lisy  wrote:
>
> > We have:
> > from module import x, y, z  # where order is not important
> >
> > Could we have something similar with ntuple (SimpleNamespace, ...)?
> >
> > maybe:
> > for x, y, z from spam:
> > print(x, y, z)
>
> What you're asking for is something like JavaScript's "object
> destructuring" syntax. It would sometimes be cool, but I haven't ever
> really yearned for it in Python. But you'd need to decide whether you
> want attributes (spam.x, spam.y) or items (spam["x"], spam["y"]). Both
> would be useful at different times.
>
>
Maybe something like

{x, y} = point

Or with aliasing

{x as dx, y as dy} = point

Then you can have

{'x' as x, 'y' as y, **} = spam


Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow function to return multiple values

[אלעזר]

What is the difference between returning a tuple and returning two values?
I think at least theoretically it's different wording for precisely the
same thing.

Elazar

בתאריך יום ה׳, 1 ביונ' 2017, 17:21, מאת Markus Meskanen ‏<
markusmeska...@gmail.com>:

> Why isn't a tuple enough? You can do automatic tuple unpack:
>
> v1, v2 = return_multiplevalues(1, 2)
>
>
> On Jun 1, 2017 17:18, "joannah nanjekye" 
> wrote:
>
> Hello Team,
>
> I am Joannah. I am currently working on a book on python compatibility and
> publishing it with apress. I have worked with python for a while we are
> talking about four years.
>
> Today I was writing an example snippet for the book and needed to write a
> function that returns two values something like this:
>
> def return_multiplevalues(num1, num2):
>  return num1, num2
>
>  I noticed that this actually returns a tuple of the values which I did
> not want in the first place.I wanted python to return two values in their
> own types so I can work with them as they are but here I was stuck with
> working around a tuple.
>
> My proposal is we provide a way of functions returning multiple values.
> This has been implemented in languages like Go and I have found many cases
> where I needed and used such a functionality. I wish for this convenience
> in python so that I don't  have to suffer going around a tuple.
>
> I will appreciate discussing this. You may also bring to light any current
> way of returning multiple values from a function that I may not know of in
> python if there is.
>
> Kind regards,
> Joannah
>
> --
> Joannah Nanjekye
> +256776468213
> F : Nanjekye Captain Joannah
> S : joannah.nanjekye
> T : @Captain_Joannah
> SO : joannah
>
>
> *"You think you know when you learn, are more sure when you can write,
> even more when you can teach, but certain when you can program." Alan J.
> Perlis*
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] fnmatch.filter_false

[אלעזר]

There shouldn't be any difference at all. Checking the for invert can be
outside of the loop, which will make the loop itself exactly as it is now.
Just like what's been done with normcase.

On Wed, May 17, 2017 at 8:55 PM  wrote:

> Top posting, apologies.
>
> I'm sure there is a better way to do it, and there is a performance hit,
> but
> its negligible.  This is also a three line delta of the function.
>
> from fnmatch import _compile_pattern, filter as old_filter
> import os
> import os.path
> import posixpath
>
>
> data = os.listdir()
>
> def filter(names, pat, *, invert=False):
> """Return the subset of the list NAMES that match PAT."""
> result = []
> pat = os.path.normcase(pat)
> match = _compile_pattern(pat)
> if os.path is posixpath:
> # normcase on posix is NOP. Optimize it away from the loop.
> for name in names:
> if bool(match(name)) == (not invert):
> result.append(name)
> else:
> for name in names:
> if bool(match(os.path.normcase(name))) == (not invert):
> result.append(name)
> return result
>
> if __name__ == '__main__':
> import timeit
> print(timeit.timeit(
> "filter(data, '__*')",
> setup="from __main__ import filter, data"
>  ))
> print(timeit.timeit(
> "filter(data, '__*')",
> setup="from __main__ import old_filter as filter, data"
> ))
>
> The first test (modified code) timed at 22.492161903402575, where the
> second
> test (unmodified) timed at 19.31892032324
>
>
>
> > -Original Message-
> > From: tritium-l...@sdamon.com [mailto:tritium-l...@sdamon.com]
> > Sent: Wednesday, May 17, 2017 1:19 PM
> > To: python-ideas@python.org
> > Subject: RE: [Python-ideas] fnmatch.filter_false
> >
> > > -Original Message-
> > > From: Python-ideas [mailto:python-ideas-bounces+tritium-
> > > list=sdamon@python.org] On Behalf Of Oleg Broytman
> > > Sent: Wednesday, May 17, 2017 12:44 PM
> > > To: python-ideas@python.org
> > > Subject: Re: [Python-ideas] fnmatch.filter_false
> > >
> > > On Wed, May 17, 2017 at 12:14:05PM -0400, Alex Walters  > > l...@sdamon.com> wrote:
> > > > Fnmath.filter works great.  To remind people what it does, it takes
> an
> > > > iterable of strings and a pattern and returns a list of the strings
> that
> > > > match the pattern.  And that is wonderful
> > > >
> > > > However, I often need to filter *out* the items that match the
> pattern
> > (to
> > > > ignore them).  In every project that I need this I end up copying the
> > > > function out of the fnmatch library and adding 'not' to the test
> clause.
> > It
> > > > would be wonderful if there was a filter_false version in the
> standard
> > > > library.  Or in inversion Boolean option.  Or something, to stop from
> > having
> > > > to copy code every time I need to ignore files.
> > >
> > >Why not create a package and publish at PyPI? Then all you need is
> > > pip install fnmatch_filter_false
> > >in your virtual env.
> >
> > That is my normal thought on something like this, but in the case of
> adding
> > a Boolean argument to fnmatch.filter, it (might be) as simple as a 3 line
> > diff that does not break the API, and as far as I can tell, does not have
> > performance implications.  Copying a module out of the standard library
> that
> > is identical except a 3 line diff that does not break compatibility with
> the
> > standard library... that just wreaks of something that should be in the
> > standard library to begin with.
> >
> > In the case of adding a separate function to fnmatch, it's still not that
> > big of a diff, and wouldn't have much duplicated code, at least in the
> way
> I
> > would implement it - it would essentially do the previous Boolean option
> > method, and wrap that.  A filter_false function, now that I think about
> it,
> > is less ideal than just adding a keyword only Boolean option to
> > fnmatch.filter.
> >
> > > Oleg.
> > > --
> > >  Oleg Broytmanhttp://phdru.name/
> p...@phdru.name
> > >Programmers don't die, they just GOSUB without RETURN.
> > > ___
> > > Python-ideas mailing list
> > > Python-ideas@python.org
> > > https://mail.python.org/mailman/listinfo/python-ideas
> > > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JavaScript-Style Object Creation in Python (using a constructor function instead of a class to create objects)

[אלעזר]

On Mon, May 15, 2017 at 6:30 PM Guido van Rossum  wrote:
> This should be worked into a PEP, instead of living on as a bunch of
python-ideas posts and blogs.
 ...
> Will someone please write a PEP?

If by "this" you mean adding to stdlib something like

@record
class Point:
x: int
y: int

or something along the lines of my "modifiers" proposal (
https://mail.python.org/pipermail//python-ideas/2016-September/042360.html),
then I think I would like to help writing such a PEP. But I thought these
proposals were rejected.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Positional-only parameters

[אלעזר]

Here's a proof-of-concept for the decorator. It does not address the issue
of passing aliases to positional arguments to **kwargs - I guess this
requires changes in the CPython's core.

(Sorry about the coloring, that's how it's pasted)

from inspect import signature, Parameter
from functools import wraps


def positional_only(n):
def wrap(f):
s = signature(f)
params = list(s.parameters.values())
for i in range(n):
if params[i].kind != Parameter.POSITIONAL_OR_KEYWORD:
raise TypeError('{} has less than {} positional
arguments'.format(f.__name__, n))
params[i] = params[i].replace(kind=Parameter.POSITIONAL_ONLY)
f.__signature__ = s.replace(parameters=params)
@wraps(f)
def inner(*args, **kwargs):
if len(args) < n:
raise TypeError('{} takes at least {} positional
arguments'.format(f.__name__, n))
return f(*args, **kwargs)
return inner
return wrap


@positional_only(2)
def f(a, b, c):
print(a, b, c)


help(f)
# f(a, b, /, c, **kwargs)

f(1, 2, c=2)

# f(1, b=2, c=3)
# TypeError: f takes at least 2 positional arguments


@positional_only(3)
def g(a, b, *, c):
print(a, b, c)

# TypeError: g has less than 3 positional arguments

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] add __contains__ into the "type" object

[אלעזר]

This suggestion is really problematic IMHO.

"isinstance" is a nominal check. I can't ask "isinstance(x, Callable[int,
int])" because that would imply solving the halting problem. so
"isinstance(x, Y)" does not mean "is it true that x is an element of the
type Y" but rather "is it true that x was created by a constructor of some
superclass of Y". It is not a type-theoretic question but a question of
origin and intent.

With regard to readability, this will be completely confusing for me. "in"
is a question about inclusion in a collection, not some set-theoretic
inclusion. Otherwise we should also as "x in f" as an equivalent to "not
not f(x)", as is in set theory.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Positional-only parameters

[אלעזר]

On Wed, Mar 1, 2017 at 9:26 PM Serhiy Storchaka  wrote:

> On 28.02.17 23:17, Victor Stinner wrote:
> > My question is: would it make sense to implement this feature in
> > Python directly? If yes, what should be the syntax? Use "/" marker?
> > Use the @positional() decorator?
>
> 


> The problem with the "/" marker is that it looks ugly. There was an
> excuse for the "*" marker -- it came from omitting the name in "*args".
> The "*" prefix itself means an iterable unpacking, but "/" is not used
> neither as prefix nor suffix.
>

I like the idea, but I wanted to note that since it has no meaning from the
point of view of the defined function, it can be done with a magic
decorator, so new syntax is not required:

@positional_only[:4]
def replace(self, old, new, count=-1):
...

It may ease googling and backporting, by defining positional_only[slice] to
be the identity function.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Settable defaulting to decimal instead of float

[אלעזר]

I think such proposals are special cases of a general theme: a compiler
pragma, similar to "from __future__", to make Python support
domain-specific syntax in the current file. Whether it's decimal literals
or matrix/vector literals etc.

I think it will be nice to make some tool, external to Python, that will
allow defining such "sibling languages" (transpiled into Python) easily and
uniformly.

Elazar

בתאריך יום ה׳, 12 בינו' 2017, 13:21, מאת Paul Moore ‏:

> On 12 January 2017 at 10:28, Victor Stinner 
> wrote:
> > George requested this feature on the bug tracker:
> > http://bugs.python.org/issue29223
> >
> > George was asked to start a discusson on this list. I posted the
> > following comment before closing the issue:
> >
> > You are not the first one to propose the idea.
>
> OK, but without additional detail (for example, how would the proposed
> flag work, if the main module imports module A, then would float
> literals in A be decimal or binary? Both could be what the user wants)
> it's hard to comment. And as you say, most of this has been discussed
> before, so I'd like to see references back to the previous discussions
> in any proposal, with explanations of how the new proposal addresses
> the objections raised previously.
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Sat, Oct 15, 2016 at 8:45 PM Chris Angelico <ros...@gmail.com> wrote:

> On Sun, Oct 16, 2016 at 4:38 AM, אלעזר <elaz...@gmail.com> wrote:
> > On Sat, Oct 15, 2016 at 8:36 PM Chris Angelico <ros...@gmail.com> wrote:
> >>
> >> On Sun, Oct 16, 2016 at 4:33 AM, אלעזר <elaz...@gmail.com> wrote:
> >> > You are confusing here two distinct roles of the parenthesis:
> >> > disambiguation
> >> > as in "(1 + 2) * 2", and tuple construction as in (1, 2, 3). This
> >> > overload
> >> > is the reason that (1) is not a 1-tuple and we must write (1,).
> >>
> >> Parentheses do not a tuple make. Commas do.
> >>
> >> 1, 2, 3, # three-element tuple
> >> 1, 2, # two-element tuple
> >> 1, # one-element tuple
> >>
> > And what [1, 2, 3] means? It's very different from [(1,2,3)].
> >
> > Python explicitly allow 1, 2, 3 to mean tuple in certain contexts, I
> agree.
> >
>
> Square brackets create a list. I'm not sure what you're not understanding,
> here.
>
> The comma does have other meanings in other contexts (list/dict/set
> display, function parameters), but outside of those, it means "create
> tuple".
>

On a second thought you may decide whether the rule is tuple and there are
exceptions, or the other way around.

The point was, conceptual expansion does not "fail" just because there is
an overloading in the meaning of the tokens ( and ). It might make it
harder to understand, though.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Sat, Oct 15, 2016 at 1:49 PM Steven D'Aprano  wrote:
...

> And the transformation of *t for the items of t (I don't care if it is a
> real transformation in the implementation, or only a fictional
> transformation) cannot work in a list comp. Let's make the number of
> items of t explicit so we don't have to worry about variable item
> counts:
>
> [*t for t in iterable]  # t has three items
> [a, b, c for (a, b, c) in iterable]
>
>
> That's a syntax error. To avoid the syntax error, we need parentheses:
>
> [(a, b, c) for (a, b, c) in iterable]
>
> and that's a no-op.


You are confusing here two distinct roles of the parenthesis:
disambiguation as in "(1 + 2) * 2", and tuple construction as in (1, 2, 3).
This overload is the reason that (1) is not a 1-tuple and we must write
(1,).

You may argue that this overloading causes confusion and make this
construct hard to understand, but please be explicit about that; even if
<1, 2,3 > was the syntax for tuples, the expansion was still

 [(a, b, c) for (a, b, c) in iterable]

Since no tuple is constructed here.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505 -- None-aware operators

[אלעזר]

On Fri, Oct 14, 2016 at 4:14 PM Gustavo Carneiro 
wrote:

> Sorry if I missed the boat, but only just now saw this PEP.
>
> Glancing through the PEP, I don't see mentioned anywhere the SQL
> alternative of having a coalesce() function:
> https://www.postgresql.org/docs/9.6/static/functions-conditional.html#FUNCTIONS-COALESCE-NVL-IFNULL
>
> In Python, something like this:
>
> def coalesce(*args):
> for arg in args:
> if arg is not None:
>  return arg
> return None
>
> Just drop it into builtins, and voila.   No need for lengthy discussions
> about which operator to use because IMHO it needs no operator.
>
> Sure, it's not as sexy as a fancy new operator, nor as headline grabbing,
> but it is pretty useful.
>
>
This has the downside of not being short-circuit - arguments to the
function are evaluated eagerly.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

בתאריך יום ו׳, 14 באוק' 2016, 12:19, מאת Michel Desmoulin ‏<
desmoulinmic...@gmail.com>:

> Regarding all those examples:
>
> Le 14/10/2016 à 00:08, אלעזר a écrit :
> > Trying to restate the proposal, somewhat more formal following Random832
> > and Paul's suggestion.
> >
> > I only speak about the single star.
> > ---
> >
> > *The suggested change of syntax:*
> >
> > comprehension ::=  starred_expression comp_for
> >
> > *Semantics:*
> >
> > (In the following, f(x) must always evaluate to an iterable)
> >
> > 1. List comprehension:
> >
> > result = [*f(x) for x in iterable if cond]
> >
> > Translates to
> >
> > result = []
> > for x in iterable:
> > if cond:
> > result.extend(f(x))
> >
> > 2. Set comprehension:
> >
> > result = {*f(x) for x in iterable if cond}
> >
> > Translates to
> >
> > result = set()
> > for x in iterable:
> > if cond:
> > result.update(f(x))
>
> Please note that we already have a way to do those. E.G:
>
> result = [*f(x) for x in iterable if cond]
>
> can currently been expressed as:
>
> >>> iterable = range(10)
> >>> f = lambda x: [x] * x
> >>> [y for x in iterable if x % 2 == 0 for y in f(x)]
> [2, 2, 4, 4, 4, 4, 6, 6, 6, 6, 6, 6, 8, 8, 8, 8, 8, 8, 8, 8]
>
>
> Now I do like the new extension syntax. I find it more natural, and more
> readable:
>
> >>> [*f(x) for x in iterable if x % 2 == 0]
>
> But it's not a missing feature, it's really just a (rather nice)
> syntaxic improvement.
>

It is about lifting restrictions from an existing syntax. That this
behavior is being *explicitly disabled* in the implementation is a strong
evidence, in my mind.

(There are more restrictions I was asked not to divert this thread, which
makes sense)

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 11:42 PM Paul Moore  wrote:

> I remain puzzled.
>
> Given the well-documented and understood transformation:
>
> [fn(x) for x in lst if cond]
>
> translates to
>
> result = []
> for x in lst:
>if cond:
>   result.append(fn(x))
>
> please can you explain how to modify that translation rule to
> incorporate the suggested syntax?
>

if you allow result.append(1, 2, 3) to mean result.extend([1,2,3])  # which
was discussed before

result = []
for x in lst:
   if cond:
  result.append(*fn(x))

Or simply use result.extend([*fn(x)])

Personally, I'm not even sure any more that I can *describe* the
> suggested syntax. Where in [fn(x) for x in lst if cond] is the *
> allowed? fn(*x)? *fn(x)? Only as *x with a bare variable, but no
> expression? Only in certain restricted types of construct which aren't
> expressions but are some variation on an unpacking construct?
>
>
The star is always exactly at the place that should "handle" it. which
means [*(fn(x)) for x in lst if cond]. fn(x) must be iterable as always.


> We've had a lot of examples. I think it's probably time for someone to
> describe the precise syntax (as BNF, like the syntax in the Python
> docs at
> https://docs.python.org/3.6/reference/expressions.html#displays-for-lists-sets-and-dictionaries
> and following sections) and semantics (as an explanation of how to
> rewrite any syntactically valid display as a loop). It'll have to be
> done in the end, as part of any implementation, so why not now?
>
>
I will be happy to do so, and will be happy to work with anyone else
interested.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 5:10 PM Steven D'Aprano  wrote:

> On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
> > About the list constructor: we construct a list by writing [a,b,c] or by
> > writing [b for b in bs]. The end result is a list
>
> I construct lists using all sorts of ways:
>

I think there is a terminology problem here (again). "Constructor" in OOP
has a specific meaning, and "constructor" in functional terminology has a
slightly different meaning. I guess Sven uses the latter terminology
because pattern matching is the dual of the constructor - it is a
"destructor" - and it feels appropriate, although admittedly confusing. In
this terminology, map(), zip() etc. are definitely not constructors. there
is only one "constructor" (list()), and there are functions that may use it
as their implementation detail. In a way, [1, 2, 3] is just a syntactic
shorthand for list construction, so it is reasonable to a call it a
constructor.

This terminology is not a perfect fit into the object-oriented world of
Python, but it is very helpful in discussion of patterns how to apply them
uniformly, since they were pretty much invented in the functional world
(ML, I think, and mathematics). One only needs to be aware of the two
different meaning, and qualify if needed, so that we won't get lost in
terminology arguments again.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 2:35 AM Steven D'Aprano <st...@pearwood.info> wrote:

> On Wed, Oct 12, 2016 at 04:11:55PM +0000, אלעזר wrote:
>
> > Steve, you only need to allow multiple arguments to append(), then it
> makes
> > perfect sense.
>
> I think you're missing a step. What will multiple arguments given to
> append do? There are two obvious possibilities:
>
> - collect all the arguments into a tuple, and append the tuple;
>
> - duplicate the functionality of list.extend
>
>
> neither of which appeals to me.
>

The latter, of course. Similar to max(). Not unheard of.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

To be honest, I don't have a clear picture of what {**x for x in d.items()}
should be. But I do have such picture for

dict(**x for x in many_dictionaries)

Elazar

‪On Wed, Oct 12, 2016 at 11:37 PM ‫אלעזר‬‎ <elaz...@gmail.com> wrote:‬

> On Wed, Oct 12, 2016 at 11:26 PM David Mertz <me...@gnosis.cx> wrote:
>
> On Wed, Oct 12, 2016 at 12:38 PM, אלעזר <elaz...@gmail.com> wrote:
>
> What is the intuition behind [1, *x, 5]? The starred expression is
> replaced with a comma-separated sequence of its elements.
>
> I've never actually used the `[1, *x, 5]` form.  And therefore, of course,
> I've never taught it either (I teach Python for a living nowadays).  I
> think that syntax already perhaps goes too far, actually; but I can
> understand it relatively easily by analogy with:
>
>  a, *b, c = range(10)
>
>
> It's not exactly "analogy" as such - it is the dual notion. Here you are
> using the "destructor" (functional terminology) but we are talking about
> "constructors". But nevermind.
>
>
> But the way I think about or explain either of those is "gather the extra
> items from the sequence." That works in both those contexts.  In contrast:
>
> >>> *b = range(10)
> SyntaxError: starred assignment target must be in a list or tuple
>
> Since nothing was assigned to a non-unpacked variable, nothing is "extra
> items" in the same sense.  So failure feels right to me.  I understand that
> "convert an iterable to a list" is conceptually available for that line,
> but we already have `list(it)` around, so it would be redundant and
> slightly confusing.
>
>
> But that's not a uniform treatment. It might have good reasons from
> readability point of view, but it is an explicit exception for the rule.
> The desired behavior would be equivalent to
>
> b = tuple(range(10))
>
> and yes, there are Two Ways To Do It. I would think it should have been
> prohibited by PEP-8 and not by the compiler. Oh well.
>
> What seems to be wanted with `[*foo for foo in bar]` is basically just
> `flatten(bar)`.  The latter feels like a better spelling, and the recipes
> in itertools docs give an implementation already (a one-liner).
>
> We do have a possibility of writing this:
>
> >>>  [(*stuff,) for stuff in [range(-5,-1), range(5)]]
> [(-5, -4, -3, -2), (0, 1, 2, 3, 4)]
>
> That's not flattened, as it should not be.  But it is very confusing to
> have `[(*stuff) for stuff in ...]` behave differently than that.  It's much
> more natural—and much more explicit—to write:
>
> >>> [item for seq in [range(-5,-1), range(5)] for item in seq]
> [-5, -4, -3, -2, 0, 1, 2, 3, 4]
>
>
> The distinction between (x) and (x,) is already deep in the language. It
> has nothing to do with this thread
>
> >>> [1, *([2],), 3]
> [1, [2], 3]
> >>> [1, *([2]), 3]
> [1, 2, 3]
>
> So there. Just like in this proposal.
>
> Elazar.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

What is the intuition behind [1, *x, 5]? The starred expression is replaced
with a comma-separated sequence of its elements.

The trailing comma Nick referred to is there, with the rule that [1,, 5] is
the same as [1, 5].

All the examples follow this intuition, IIUC.

Elazar

בתאריך יום ד׳, 12 באוק' 2016, 22:22, מאת David Mertz ‏:

> I've followed this discussion some, and every example given so far
> completely mystifies me and I have no intuition about what they should mean.
>
> On Oct 12, 2016 8:43 AM, "Steven D'Aprano"  wrote:
>
> On Tue, Oct 11, 2016 at 02:42:54PM +0200, Martti Kühne wrote:
> > Hello list
> >
> > I love the "new" unpacking generalisations as of pep448. And I found
> > myself using them rather regularly, both with lists and dict.
> > Today I somehow expected that [*foo for foo in bar] was equivalent to
> > itertools.chain(*[foo for foo in bar]), which it turned out to be a
> > SyntaxError.
>
> To me, that's a very strange thing to expect. Why would you expect that
> unpacking items in a list comprehension would magically lead to extra
> items in the resulting list? I don't think that makes any sense.
>
> Obviously we could program list comprehensions to act that way if we
> wanted to, but that would not be consistent with the ordinary use of
> list comprehensions. It would introduce a special case of magical
> behaviour that people will have to memorise, because it doesn't follow
> logically from the standard list comprehension design.
>
> The fundamental design principle of list comps is that they are
> equivalent to a for-loop with a single append per loop:
>
> [expr for t in iterable]
>
> is equivalent to:
>
> result = []
> for t in iterable:
> result.append(expr)
>
>
> If I had seen a list comprehension with an unpacked loop variable:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
>
> I never in a million years would expect that running a list
> comprehension over a three-item sequence would magically expand to six
> items:
>
> [1, 'a', 2, 'b', 3, 'c']
>
>
> I would expect that using the unpacking operator would give some sort
> of error, or *at best*, be a no-op and the result would be:
>
> [(1, 'a'), (2, 'b'), (3, 'c')]
>
>
> append() doesn't take multiple arguments, hence a error should be the
> most obvious result. But if not an error, imagine the tuple unpacked to
> two arguments 1 and 'a' (on the first iteration), then automatically
> packed back into a tuple (1, 'a') just as you started with.
>
> I think it is a clear, obvious and, most importantly, desirable property
> of list comprehensions with a single loop that they cannot be longer
> than the initial iterable that feeds them. They might be shorter, if you
> use the form
>
> [expr for t in iterable if condition]
>
> but they cannot be longer.
>
> So I'm afraid I cannot understand what reasoning lead you to
> expect that unpacking would apply this way. Wishful thinking
> perhaps?
>
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

Steve, you only need to allow multiple arguments to append(), then it makes
perfect sense.

בתאריך יום ד׳, 12 באוק' 2016, 18:43, מאת Steven D'Aprano ‏<
st...@pearwood.info>:

> On Tue, Oct 11, 2016 at 02:42:54PM +0200, Martti Kühne wrote:
> > Hello list
> >
> > I love the "new" unpacking generalisations as of pep448. And I found
> > myself using them rather regularly, both with lists and dict.
> > Today I somehow expected that [*foo for foo in bar] was equivalent to
> > itertools.chain(*[foo for foo in bar]), which it turned out to be a
> > SyntaxError.
>
> To me, that's a very strange thing to expect. Why would you expect that
> unpacking items in a list comprehension would magically lead to extra
> items in the resulting list? I don't think that makes any sense.
>
> Obviously we could program list comprehensions to act that way if we
> wanted to, but that would not be consistent with the ordinary use of
> list comprehensions. It would introduce a special case of magical
> behaviour that people will have to memorise, because it doesn't follow
> logically from the standard list comprehension design.
>
> The fundamental design principle of list comps is that they are
> equivalent to a for-loop with a single append per loop:
>
> [expr for t in iterable]
>
> is equivalent to:
>
> result = []
> for t in iterable:
> result.append(expr)
>
>
> If I had seen a list comprehension with an unpacked loop variable:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
>
> I never in a million years would expect that running a list
> comprehension over a three-item sequence would magically expand to six
> items:
>
> [1, 'a', 2, 'b', 3, 'c']
>
>
> I would expect that using the unpacking operator would give some sort
> of error, or *at best*, be a no-op and the result would be:
>
> [(1, 'a'), (2, 'b'), (3, 'c')]
>
>
> append() doesn't take multiple arguments, hence a error should be the
> most obvious result. But if not an error, imagine the tuple unpacked to
> two arguments 1 and 'a' (on the first iteration), then automatically
> packed back into a tuple (1, 'a') just as you started with.
>
> I think it is a clear, obvious and, most importantly, desirable property
> of list comprehensions with a single loop that they cannot be longer
> than the initial iterable that feeds them. They might be shorter, if you
> use the form
>
> [expr for t in iterable if condition]
>
> but they cannot be longer.
>
> So I'm afraid I cannot understand what reasoning lead you to
> expect that unpacking would apply this way. Wishful thinking
> perhaps?
>
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add "equal" builtin function

[אלעזר]

On Thu, Oct 6, 2016 at 5:53 PM Sjoerd Job Postmus <sjoerd...@sjoerdjob.com>
wrote:

> On Thu, Oct 06, 2016 at 02:45:11PM +0000, אלעזר wrote:
> > It is a real problem. People are used to write `seq == [1, 2, 3]` and it
> > passes unnoticed (even with type checkers) that if seq changes to e.g. a
> > tuple, it will cause subtle bugs. It is inconvenient to write `len(seq)
> ==
> > 3 and seq == [1, 2, 3]` and people often don't notice the need to write
> it.
> >
> > (I'd like to note that it makes sense for this operation to be written as
> >
> > *iter1 == *lst
> >
> > although it requires a significant change to the language, so a
> > Sequence.equal function makes sense)
> >
> > Elazar
> >
>
> I think you're mistaken about the suggestion.


You are right of course. Sorry.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add "equal" builtin function

[אלעזר]

It is a real problem. People are used to write `seq == [1, 2, 3]` and it
passes unnoticed (even with type checkers) that if seq changes to e.g. a
tuple, it will cause subtle bugs. It is inconvenient to write `len(seq) ==
3 and seq == [1, 2, 3]` and people often don't notice the need to write it.

(I'd like to note that it makes sense for this operation to be written as

*iter1 == *lst

although it requires a significant change to the language, so a
Sequence.equal function makes sense)

Elazar

On Thu, Oct 6, 2016 at 5:02 PM Paul Moore  wrote:

> On 6 October 2016 at 14:45, Filipp Bakanov  wrote:
> > For now there are many usefull builtin functions like "any", "all", etc.
> I'd
> > like to propose a new builtin function "equal". It should accept
> iterable,
> > and return True if all items in iterable are the same or iterable is
> emty.
> > That's quite popular problem, there is a discussion of how to perform it
> on
> > stackoverflow
> > (
> http://stackoverflow.com/questions/3844801/check-if-all-elements-in-a-list-are-identical
> )
> > - all suggestions are either slow or not very elegant.
> > What do you think about it?
>
> It's not a problem I've needed to solve often, if at all (in
> real-world code). But even if we assume it is worth having as a
> builtin, what would you propose as the implementation? The
> stackoverflow discussion highlights a lot of approaches, all with
> their own trade-offs. One problem with a builtin is that it would have
> to work on all iterables, which is likely to preclude a number of the
> faster solutions (which rely on the argument being an actual list).
>
> It's an interesting optimisation problem, and the discussion gives
> some great insight into how to micro-optimise an operation like this,
> but I'd question whether it needs to be a language/stdlib feature.
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Improve error message when missing 'self' in method definition

[אלעזר]

Isn't it possible to implement it as a pure Python exception hook?

On Wed, Oct 5, 2016 at 10:04 PM Ivan Levkivskyi 
wrote:

>
> On 5 October 2016 at 20:55, Yury Selivanov 
> wrote:
>
>
> Speaking of, I'm not much of a C hacker, and messing with CPython internals
> is a little daunting. If anyone wants to take this on, you have my
> blessing. I also may take a shot at implementing this idea in the next
> couple weeks when I have some time.
>
>
> It would help if you could create an issue and write exhaustive unittests
> (or at least specifying how exactly the patch should work for all corner
> cases).  Someone with the knowledge of CPython internals will later add the
> missing C code to the patch.
>
> Yury
>
>
> I agree with Yury here. There are corner cases (like what to do with
> classmethods etc). If behaviour for all of them are specified, it would be
> quite straightforward to implement this.
>
> --
> Ivan
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Clear screen command for the REPL

[אלעזר]

"Bash on Ubuntu on windows" responds to CTRL+D just fine. I don't really
know how it works, but it looks like it is based on the Windows terminal
emulator.

Elazar

בתאריך יום ה׳, 29 בספט' 2016, 06:36, מאת Chris Angelico ‏:

> On Thu, Sep 29, 2016 at 12:04 PM, Steven D'Aprano 
> wrote:
> > (Also, it seems a shame that Ctrl-D is EOF in Linux and Mac, but Windows
> > is Ctrl-Z + Return. Can that be standardized to Ctrl-D everywhere?)
>
> Sadly, I suspect not. If you're running in the default Windows
> terminal emulator (the one a normal user will get by invoking
> cmd.exe), you're running under a lot of restrictions, and I believe
> one of them is that you can't get Ctrl-D without an enter.
>
> But it would be very nice to support both.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

Thank you all. I think this thread is pretty much close by now. I
understand at least most of your concerns and I will take time to shape my
idea.

I wanted to note one last thing, though, regarding my claim that
annotations are not actually standard expressions: Guido had once expressed
his concerns regarding the performance hit of using cast(), since it is not
easily (or at all) optimized away. This performance hit should not be there
in the first place, if the distinction between annotations and evaluatable
expressions was kept - i.e. by allowing the attachment of annotations to
expressions (as I believe was proposed several times).

Now, I understand that there are very good reasons not to allow it; keeping
the language simple and familiar would be my first guess - but note how the
semantics of the "main" language is hindered by the complexities of its
specification-related syntactic subset, which is not due, in my opinion. If
you want to specify things, the syntactic hit is unavoidable, but the
semantic hit is not.

(BTW why isn't it written cast[T](exp) ?)

Thank you again for this discussion
Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Sun, Sep 25, 2016 at 10:14 PM David Mertz  wrote:

> All of those might promote changes in tools. But the tools exist to aid
> working with a language, not the other way around.
>
> I can't think of a way to know that this string is code and that string is
an actual string. And figuring out means finding context dependency.

I would not want my editor to do that much highlighting distinction, but
> assuming you do, all your comments defeat your own point. If an expression
> being within an annotation merits different evaluation semantics, it is a
> vastly smaller change to say a string being in an annotation merits
> different syntax highlighting.
>
It is a smaller change, but of a very different kind. I think
annotations *theoretically* deserve different syntax (e.g. using the arrow
notation) but keeping it an Expression keeps things simpler. Just like
type(type) is not type, *theoretically*, but merging these objects is a
very smart decision.

So yes, I think it should have (and already has) different semantics; but
its useful to have the same syntax (and related semantics). Simple
keyword-highlighting "except inside quotes" should still be very useful.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Sun, Sep 25, 2016 at 9:28 PM David Mertz <me...@gnosis.cx> wrote:

> On Sep 25, 2016 10:59 AM, "אלעזר" <elaz...@gmail.com> wrote:
> > 2. It is not naturally supported by syntax highlighters and IDEs. They
> can be made to support it, but most will not.
>
> This is a complete red herring. Having a highlight rule of "apply
> highlights in string annotations" is straightforward in modern editors.
> This is like arguing Python should do  because Notepad.exe
> doesn't do something smart with it.
>

Not that I think it's a killer argument, but why a red herring? Quick
search does not find such an explicit option in Gedit, PyDev and yes,
Notepad++.exe. It is not a common or default option.

Having such a rule by default amounts to admitting that these are not
essentially strings, and the quotes there are overloaded. It also means
that actual strings are not understood as such, and are incorrectly
highlighted. But please let's not delve into this: it is of some
importance, but should not affect an actual decision.

IDEs are more important. Renaming facilities do over-renaming or
under-renaming because of this need to rename inside some strings, but not
inside others. Similarly code search facilities, and warnings from IDEs
about inlining variables. I have encountered real bugs caused by such an
imperfect renaming (and I hope your answer is not "don't do renaming").

A prefix like

  code"foo()"

might help of course, but it is not really used as a string.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

Thank you Stephen. You have phrased my proposal better than I did.

As per the using quoted strings, the problems are:
1. The well-formedness of the expression is not checked by the compiler.
2. It is not naturally supported by syntax highlighters and IDEs. They can
be made to support it, but most will not. Partly because
3. There is no natural way to distinguish quoted expressions from actual
human-readable text (as in the begins library).
4. (My own taste): this is ugly and inconsistent, and there's 2 meaningless
characters there :) (6 if multiline)


On Sun, Sep 25, 2016 at 8:42 PM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Nick Coghlan writes:
>
>  > This code works as a doctest today:
>  >
>  > >>> def func(a: "Expected output"):
>  > ... pass
>  > ...
>  > >>> print(func.__annotations__["a"])
>  > Expected output
>  >
>  > Any change that breaks that currently valid doctest is necessarily a
>  > compatibility break for the way annotations are handled at runtime. It
>  > doesn't matter for that determination how small the change to fix the
>  > second command is, it only matters that it *would* have to change in
>  > some way.
>
> This is a bit unfair to אלעזר, although it's been a long thread so I
> can understand why some of his ideas have gone missing.  His proposals
> have gotten a bit incoherent because he has been answering all the
> different objections one by one rather than organizing things into a
> single design, but I think eventually he would organize it as follows:
>
> (1) Add __raw_annotations__ and save the thunked expressions there,
> whether as code objects or AST.
> (2) Turn __annotations__ into a property which evaluates (and
> memoizes?) the thunks and returns them.  (First explicitly
> suggested by Alexander Belopol, I think.)
>
> He claims that (2) solves the backward compatibility problem, I don't
> have the knowledge to figure out whether it is that simple or not.  It
> seems plausible to me, so I'd love to hear an explanation.  New ideas
> like DBC would of course be supported by the new __raw_annotations__
> since there's no backward compatibility issue there.
>
> I'm still -1 on the whole thing on the principle "although sometimes
> never is better than *right* now". I think the aClass = "aClass" trick
> described by Chris is perfectly serviceable to deal with the PEP 484
> forward type reference issue.  The "let's turn all the annotations
> into expressions" idea can be practically exploited with ast.parse().
> I'm guessing a decorator could be used to provide __raw_annotations__
> and __annotations__ per (1) and (2) above (although I'm not sure how
> to do it myself: copying __annotations__ to __raw_annotations__ and
> then propertizing __annotations__ could be a bit tricky, I guess).
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

I promised not to bother you, but I really can't. So here's what I felt I
have to say. This email is quite long. Please do not feel obliged to read
it. You might find some things you'll want to bash at the end though :)

Short-ish version:

1. Please consider disallowing the use of side effects of any kind in
annotations, in that it is not promised when it will happen, if at all. So
that a change 3 years from now will be somewhat less likely to break
things. Please consider doing this for version 3.6; it is feature-frozen,
but this is not (yet) a feature, and I got the feeling it is hardly
controversial.

I really have no interest in wasting the time of anybody here. If this
request is not something you would ever consider, please ignore the rest of
this email.

2. A refined proposal for future versions of the language: the ASTs of the
annotation-expressions will be bound to __raw_annotations__.
   * This is actually more in line to what PEP-3107 was about ("no assigned
semantics"; except for a single sentence, it is only about expressions. Not
objects).
   * This is helpful even if the expression is evaluated at definition
time, and can help in smoothing the transformation.

3. The main benefit from my proposal is that contracts (examples,
explanations, assertions, and types) are naturally expressible as (almost)
arbitrary Python expressions, but not if they are evaluated or evaluatable,
at definition time, by the interpreter. Why: because it is really written
in a different language - *always*. This is the real reason behind the
existence, and the current solutions, of the forward reference problem. In
general it is much more flexible than current situation.

4. For compatibility, a new raw_annotations() function will be added, and a
new annotations() function will be used to get the eval()ed version of
them. Similarly to dir(), locals() and globals().
  * Accessing __annotations__ should work like calling annotations(), but
frowned upon, as it might disappear in the far future.
  * Of course other `inspect` functions should give the same results as
today.
  * Calling annotations()['a'] is like a eval(raw_annotations()['a']) which
resembles eval(raw_input()).

I believe the last point has a very good reason, as explained later: it is
an interpretation of a different language, foreign to the interpreter,
although sometimes close enough to be useful. It is of course well formed,
so the considerations are not really security-related.

I am willing to do any hard work that will make this proposal happen
(evaluating existing libraries, implementing changes to CPython, etc) given
a reasonable chance for acceptance.

Thank you,
Elazar

---

Long version:

Stephen - I read your last email only after writing this one; I think I
have partially addressed the lookup issue (with ASTs and scopes), and
partially agree: if there's a problem implementing this feature, I should
look deeper into it. But I want to know that it _might_ be considered
seriously, _if_ it is implementable. I also think that Nick refuted the
claim that evaluation time and lookup *today* are so simple to explain. I
know I have hard time explaining them to people.

Nick, I have read your blog post about the high bar required for
compatibility break, and I follow this mailing list for a while. So I agree
with the reasoning (from my very, very little experience); I only want to
understand where is this break of compatibility happen, because I can't see
it.

Chris:

On Fri, Sep 23, 2016 at 6:59 PM Chris Angelico <ros...@gmail.com> wrote:

> On Fri, Sep 23, 2016 at 11:58 PM, אלעזר <elaz...@gmail.com> wrote:

> "Unknown evaluation time" is scary. _for expressions_, which might have
> side
> > effects (one of which is running time). But annotations must be pure by
> > convention (and tools are welcome to warn about it). I admit that I
> propose
> > breaking the following code:
> >
> > def foo(x: print("defining foo!")): pass
> >
> > Do you know anyone who would dream about writing such code?
>
> Yes, side effects make evaluation time scary. But so do rebindings,
> and any other influences on expression evaluation. Good, readable code
> generally follows the rule that the first instance of a name is its
> definition.


No, it isn't. I guess that even the code you write or consider to be
excellent and readable still contains functions that use entities defined
only later in the code. It is only when you follow execution path that you
should be already familiar with the names.

I think rebinding is only scary when it is combined with side effect or
when the name lookup is not clear. And why do you call it _re_binding?


> >> > > class MyClass:
> >> > > pass
> >> > >
> >> > > def function(arg: MyCalss):
> >> > > ...
> >> > >
> >

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

Thank you all for your feedback. I will try to respond to it in a way that
will not waste your time, But to do that I still need an example for the
stromgest issue raised - bacwards compatibility. It is not just the mere
change that is incompatible, since _any_ visible change is incompatible in
some way, or otherwise it wasn't visible.

Again, I assume ".__annotations__" access evaluates them in the original
context.

I couldn't find any useful example yet.

Elazar

בתאריך שבת, 24 בספט' 2016, 22:07, מאת Stephen J. Turnbull ‏<
turnbull.stephen...@u.tsukuba.ac.jp>:

> אלעזר writes:
>
>  > But _some_ people (medium-level, Steven, whose main language is
>  > probably not Python) will not even know [function names are looked
>  > up at runtime, and so the called function may not be why you think
>  > it is] is the case.
>
> And the dinosaurs will have returned by independent evolution by the
> time it matters to them (unless it's a deliberate attack, in which
> case people at that level would be toast anyway).
>
> But I think you're completely missing what people are trying to tell
> you.  You shouldn't be so concerned with refuting their arguments
> because it doesn't matter.  No matter how many points you amass for
> technique, you're going to get creamed on style points anyway.  It's
> like this:
>
> (1) Python is a "consenting adults" language, and that is presumed by
> its development culture.  The goal is not to stop people from
> creating "functions that look like recursions but aren't" on
> purpose; it's to make it easy for them to write recursive
> functions if they want to.  From your example, that goal is
> obviously satisfied.  Nobody who matters wants to go farther than
> that in Python.
>
> The reason one can create "functions that look like recursions but
> aren't" is because another of Python's goals is to ensure that all
> things -- specifically including functions -- are objects that can
> be manipulated "the same way" where appropriate -- in this case,
> saving off the original function object somewhere then rebinding
> the original name to something else.[1]  Granted, we don't go so
> far as Lisp where expressions are lists that you can manipulate
> like any other list, but aside from the fact that the code itself
> is an opaque object, functions are no different from other
> objects.  Even builtins:
>
> Python 3.6.0a4 (default, Sep  3 2016, 19:21:32)
> >>> def help(*ignored, **paid_no_attention):
> ...  print("Ouch, you just shot off your foot!")
> ...
> >>> help(help)
> Ouch, you just shot off your foot!
> >>>
>
> Shooting off your opposite extremity by redefining builtin classes
> is left as an exercise for the reader.
>
> All of this is a matter of the general attitude of pragmatism and
> bias toward simplicity of implementation (both enshrined in the
> Zen of Python).
>
> (2) You keep talking about others being lost in terminology, but in
> the context of Python discussions, you have a really big problem
> yourself.  You use the phrase "just an annotation" as though that
> means something, but there is nothing like a "just an "
> in Python discourse, not in the sense that "once we introduce
> s, they can be anything we want".  The Language
> Reference defines what things are possible, and truly new ones are
> rarely added.
>
> This is deliberate.  Another design principle is Occam's Razor,
> here applied as "new kinds of thing shall not spring up like
> whiskers on Barry's chin."  Yes, function annotations need new
> syntax and so are a new kind of thing to that extent.  *Their
> values don't need to be,* and even the annotations themselves are
> implemented in the preferred way for "new things" (a dunder on an
> existing type).  Since it's new syntax, it's language-level, and
> so the values are going to be something already defined in the
> language reference.  "Expression resolving to object to be saved
> in an attribute on the function" seems to be as close to "anything
> you want" as you're gonna get without a new kind of thing.
>
> (3) Python has a very simple model of expressions.  The compiler turns
> them into code.  The interpreter executes that code, except in the
> case where it is "quoted" by the "def" or "lambda" keywords, in
> which case it's stored in an object (and in the case of "def",
> registered in a namespace).
>
> 

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 3:11 PM Steven D'Aprano <st...@pearwood.info> wrote:

> On Fri, Sep 23, 2016 at 10:17:15AM +0000, אלעזר wrote:
> > On Fri, Sep 23, 2016 at 6:06 AM Steven D'Aprano <st...@pearwood.info>
> wrote:
> > > On Thu, Sep 22, 2016 at 07:21:18PM +, אלעזר wrote:
> > > > On Thu, Sep 22, 2016 at 9:43 PM Steven D'Aprano wrote:
> > > > > On Thu, Sep 22, 2016 at 05:19:12PM +, אלעזר wrote:
> > > > > > Hi all,
> > > > > >
> > > > > > Annotations of function parameters and variables are evaluated
> when
> > > > > > encountered.
> > > > >
> > > > > Right, like all other Python expressions in general, and
> specifically
> > > > > like function parameter default arguments.
> > > > >
> > > >
> > > > Just because you call it "expression", when for most purposes it
> isn't -
> > > > it is an annotation.
> > >
> > > It is *both*. It's an expression, because it's not a statement or a
> > > block.
> >
> >
> > Did you just use a false-trichotomy argument? :)
>
> No.
>
> You are the one trying to deny that annotations are expressions -- I'm
> saying that they are both annotations and expressions at the same time.
> There's no dichotomy here, since the two are not mutually exclusive.
> (The word here is dichotomy, not trichotomy, since there's only two
> things under discussion, not three.)
>
>
The argument "It's an expression, because it's not a statement or a block"
assumes that things must an expression, a statement or a block. Hence
"trichotomy". And it is false.

But I think we are getting lost in the terminology. Since I propose no
change in what is considered valid syntax,


> > > You cannot write:
> > >
> > > def func(arg1: while flag: sleep(1), arg2: raise ValueError):
> > > ...
> >
> > > because the annotation must be a legal Python expression, not a code
> > > block or a statement.
> >
> >
> > This is the situation I'm asking to change
>
> That's a much bigger change than what you suggested earlier, changing
> function annotations to lazy evaluation instead of eager.
>
> Supporting non-expressions as annotations -- what's your use-case? Under
> what circumstances would you want to annotate an function parameter with
> a code block instead of an expression?
>
>
It indeed came out different than I meant. I don't suggest allowing
anything that is not already allowed, syntactically. I only propose giving
the current syntax a slightly different meaning, in a way that I'm sure
matches how Python coders already understand the code.

> > It's an annotation because that's the
> > > specific *purpose* of the expression in that context.
> >
> > Exactly! Ergo, this is an annotation.
>
> I've never denied that annotations are annotations, or that annotations
> are used to annotate function parameters. I'm not sure why you are
> giving a triumphant cry of "Exactly!" here -- it's not under dispute
> that annotations are annotations.
>
>
:( this kind of fighting over terminology takes us nowhere indeed.

What other context you see where the result of an expression is not
intended to be used at all? Well there's Expression statements, which are
evaluated for side effect. There's docstrings, which are a kind of
annotations. What else? The only other that comes to mind is
reveal_type(exp)... surely I don't need evaluation there.

And it shouldn't be under dispute that annotations are expressions.
> They're not code blocks. They're not statements. What else could they be
> apart from expressions?
>
> Now it is a false dichotomy as question. The answer is "annotation" as an
independent concept, closely related to expressions, but not evaluated when
encountered. Very similar to E in `lambda: E` except that lambda are there
mainly for the resulting value (hence "expression") and annotations are
there mainly for being there. In the code.


> The PEP that introduced them describes them as expressions:
>
> Function annotations are nothing more than a way of associating
> arbitrary Python EXPRESSIONS with various parts of a function at
> compile-time. [Emphasis added.]
>
> https://www.python.org/dev/peps/pep-3107/
>
>
Syntactically, yes. Just like X in "a = lambda: X" is an expression, but
you don't see it evaluated, do you? And this is an _actual_ expression,
undeniably so, that is intended to be evaluated and used at runtime.


> and they are documented as an expression:
>
> parameter ::=  identifier [":" exp

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 6:24 AM Nick Coghlan <ncogh...@gmail.com> wrote:

> On 23 September 2016 at 13:06, Steven D'Aprano <st...@pearwood.info>
> wrote:
> > On Thu, Sep 22, 2016 at 07:21:18PM +, אלעזר wrote:
> >> "Expression" is something that you need its value right
> >> now, and "annotation" is something that, well, annotates the code you
> see
> >> right now.
> >
> > Right. In the case of Python, function annotations **do** have a runtime
> > effect: the expressions are evaluated, and the evaluated results are
> > assigned in function.__annotations__ and made available for runtime
> > introspection.
> >
> > Don't think that function annotations are **only** for the static type
> > checker. Python is a much richer language than that!
>
> If folks are after a simple non-type-checking related example of
> annotation usage, the "begins" CLI library is a decent one:
> https://pypi.python.org/pypi/begins
>
> That lets you supply command line help for parameters as annotations:
>
> 
> In Python3, any function annotations for a parameter become the
> command line option help. For example:
>
>   >>> import begin
>   >>> @begin.start # doctest: +SKIP
>   ... def run(name: 'What, is your name?',
>   ... quest: 'What, is your quest?',
>   ... colour: 'What, is your favourite colour?'):
>   ... pass
>
> Will generate command help like:
>
> usage: holygrail_py3.py [-h] -n NAME -q QUEST -c COLOUR
>
> optional arguments:
>   -h, --helpshow this help message and exit
>   -n NAME, --name NAME  What, is your name?
>   -q QUEST, --quest QUEST
> What, is your quest?
>   -c COLOUR, --colour COLOUR
> What, is your favourite colour?
> 
>
> It's not a substitute for something like click or argparse when it
> comes to more complex argument parsing, but it's a good example of the
> kind of simple pseudo-DSL folks have long been able to create with
> annotations independently of the type hinting use case.
>
>
That's a very nice use, and I was wrong - I did know it; I've found it not
long ago when I wanted to implement it myself...

And guess what? It does not require eager evaluation _at all_. No
decorator-helped-annotation mechanism require eager evaluation built into
the language. Lazy evaluation is more general than eager, in that it can
always be forced (and not the other way around).

def eager_annotation(f):
f.__annotations__ = {k:v() for k, v in f.__annotations__}
return f

Use @eager_annotation wherever you like, or collapse it into other
decorators.

You don't need @eager_annotation for type annotations, or any other form of
annotation without runtime semantics. On the other hand - if you do want
side effect in this function's annotations, well there's better be some
nice big @EAGER! decorator above it.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 6:06 AM Steven D'Aprano <st...@pearwood.info> wrote:

> On Thu, Sep 22, 2016 at 07:21:18PM +0000, אלעזר wrote:
> > On Thu, Sep 22, 2016 at 9:43 PM Steven D'Aprano <st...@pearwood.info>
> wrote:
> >
> > > On Thu, Sep 22, 2016 at 05:19:12PM +, אלעזר wrote:
> > > > Hi all,
> > > >
> > > > Annotations of function parameters and variables are evaluated when
> > > > encountered.
> > >
> > > Right, like all other Python expressions in general, and specifically
> > > like function parameter default arguments.
> > >
> >
> > Just because you call it "expression", when for most purposes it isn't -
> it
> > is an annotation.
>
> It is *both*. It's an expression, because it's not a statement or a
> block.


Did you just use a false-trichotomy argument? :)


> You cannot write:
>
> def func(arg1: while flag: sleep(1), arg2: raise ValueError):
> ...

because the annotation must be a legal Python expression, not a code
> block or a statement.


This is the situation I'm asking to change


> It's an annotation because that's the
> specific *purpose* of the expression in that context.
>
>
Exactly! Ergo, this is an annotation.


> As an analogy: would you argue that it is wrong to call the for-loop
> iterable an expression?
>
> for  in :
> block
>
> I trust that you understand that the loop iterable can be any expression
> that evaluates to an iterable. Well, annotations can be any expression
> that evaluates to anything at all, but for the purposes of type
> checking, are expected to evaluate to a string or a type object.
>
>
for-loop iterable is an expression, evaluated at runtime, _for_ the
resulting value to be used in computation. A perfectly standard expression.
Nothing fancy.


> In the case of function annotations, remember that they can be any
> legal Python expression. They're not even guaranteed to be type
> annotations. Guido has expressed a strong preference that they are only
> used as type annotations, but he hasn't yet banned other uses (and I
> hope he doesn't), so any "solution" for a type annotation problem must
> not break other uses.
>
>
Must *allow* other use cases. My proposal allows: just evaluate them at the
time of their use, instead at definition time.


>
> > "Expression" is something that you need its value right
> > now, and "annotation" is something that, well, annotates the code you see
> > right now.
>
> Right. In the case of Python, function annotations **do** have a runtime
> effect: the expressions are evaluated, and the evaluated results are
> assigned in function.__annotations__ and made available for runtime
> introspection.
>
> Don't think that function annotations are **only** for the static type
> checker. Python is a much richer language than that!
>
>
function.__annotations__  can have the delayed value, be it a lambda, ast
or string. It can also be computed at the time of access as suggested
earlier.


>
> > > > It is
> > > > also easy to forget, and the result might be a (very uninteresting)
> > > > exception in certain untested paths, e.g. inside functions.
> > >
> > > Unlikely, unless you're talking about functions nested inside other
> > > functions, or unusual (but legal and sometimes useful) conditional
> > > definitions:
> >
> > I was thinking about the former, but yeah, uncovered code will fail at
> > runtime, possibly in production, for *no* real reason. I do not claim
> that
> > this is common, but it is definitely unnecessary - unlike initialization
> > expressions.
>
> Unnecessary?
>
> class MyClass:
> pass
>
>
> def function(arg: MyCalss):
> ...
>
> I want to see an immediate NameError here, thank you very much
>

Two things to note here:
A. IDEs will point at this NameError
B. Type checkers catch this NameError
C. Even the compiler can be made to catch this name error, since the name
MyCalss is bound to builtins where it does not exist - you see, name lookup
does happen at compile time anyway. I'm not really suggesting the compiler
should make it error though.
D. Really, where's the error here? if no tool looks at this signature,
there's nothing wrong with it - As a human I understand perfectly. If a
tool will look at it, it will warn or fail, exactly as I would liked it too.

function.__annotations__['arg']()
>
> to see whether or not the annotation is valid.
>
> I accept that using strings as forward annotations is not a foolproof
> solution either:
>
>
> def function(arg: 'MyCalss'):
> ...
>
but let's not jump 

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 5:54 AM Chris Angelico  wrote:

> On Fri, Sep 23, 2016 at 12:35 PM, Steven D'Aprano 
> wrote:
> > The straight-forward and simple way of writing a recursive spam()
> > function surprises beginners, but they might go years or their entire
> > career without running into a situation where they are caught by
> > surprise. After all, it is rare for productuon code to rename functions,
> > and rarer still to do it to recursive functions:
> >
> > func = spam
> > spam = something_else()
> > func()  # why does the recursion not work???
> >
> > In production code, that sort of thing almost never happens.
>
> There's actually one very common technique involving rebinding functions.
>
> @count_calls
> def mergesort(lst):
> mid = len(lst) // 2
> if not mid: return lst
> return merge(mergesort(lst[..mid]), mergesort(lst[mid..]))
>
> *Obviously* this is recursive. But if you used some magic that said
> "call the function that's currently being called", you'd be bypassing
> the count_calls decoration (which would presumably work by creating a
> wrapper function). Yes, it may defeat some potential optimizations (eg
> tail recursion optimization), but it enables all this flexibility.
>
> So we _need_ to have this kind of rebind available, and not just for
> experts.
>
>
I think you are mixing levels of abstraction because you know how this is
implemented. The user only sees "A function named mergesort decorated by
count_calls". She does not see "A function named mergesort passed to a
higher order function named count_calls whose result is bound into the
variable mergesort". Even if the latter is exactly what happens,
declaratively the former is more accurate by intention.

Ideally, the calls to mergesort will rebind to this _decorated_ function.
not to the mutable global variable. Again, the argument that it will be
very hard to implement it in a different way, or that is will break things,
is a very strong argument, and I am not confronting it.

> In the meantime, I'll usually just write my recursive functions the
> > old-fashioned normal way.
>
> As will I. Of course, people are welcome to work differently, just as
> long as I never have to write tests for their code, or refactor
> anything into a decorator, or anything like that. I want the
> POWAH! :)
>

As will I, simply because the old-fashioned way is more readable. And I
will sadly accept the fact that I can't be 100% sure what's function is
called at runtime. But _some_ people (medium-level, Steven, whose main
language is probably not Python) will not even know this is the case.

Tests are important and could have reworked into the system (through
inspect, or by using a special import which allow monkey patching). I can't
see why the ability to test must remain in production.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 4:17 AM Nick Coghlan  wrote:
...

> As others have noted, the general idea of allowing either a
> placeholder name or the class name to refer to a suitable type
> annotation is fine, though - that would be a matter of implicitly
> injecting that name into the class namespace after calling
> __prepare__, and ensuring the compiler is aware of that behaviour,
> just as we inject __class__ as a nonlocal reference into method bodies
> that reference "super" or "__class__".
>
> Just to be sure I understand, will the following work?

class A:
def repeat(n: int) -> List[A]: pass

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 12:18 AM Chris Angelico  wrote:

> # Recursion in functions
> def spam():
> return spam()
>

I just note that it *is* surprising, for most users, that you can't be sure
that this is a recursion, yet. So it if you want a trusted-upon recursion
you should write

# spam:
def spam():
def spam():
return spam()
return spam()


Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Fri, Sep 23, 2016 at 12:05 AM Ivan Levkivskyi <levkivs...@gmail.com>
wrote:

> On 22 September 2016 at 22:02, אלעזר <elaz...@gmail.com> wrote:
>
>> On Thu, Sep 22, 2016 at 10:58 PM David Mertz <me...@gnosis.cx> wrote:
>>
>>> On Thu, Sep 22, 2016 at 12:35 PM, אלעזר <elaz...@gmail.com> wrote:
>>>
>>>> I think we're talking about different things here. I just referred to
>>>> the common need to use the name of the current class in type annotation
>>>>
>>>> class A:
>>>> def add(self, other: A) -> A: ...
>>>>
>>>
>>> Yeah, I find the need for using the string "A" here a wart. Rather than
>>> change the entire semantics of annotations, it feels like a placeholder for
>>> this meaning would be better.  E.g.:
>>>
>>> class A:
>>> def __add__(self, other: CLS) -> CLS: ...
>>>
>>> A static checker could do the magic of recognizing that special name
>>> easily enough (no harder than recognizing the quoted string).  At runtime
>>> 'CLS' could either just be a singleton with no other behavior... or perhaps
>>> it could be some sort of magic introspection object.  It's more verbose,
>>> but you can also spell it now as:
>>>
>>> class A:
>>> def __add__(self, other: type(self)) -> type(self): ...
>>>
>>> That's a little ugly, but it expresses the semantics we want.
>>>
>>
> Concerning the __add__ method, I think a more typical type for it is
>
> T = TypeVar('T', bound='A')
>
> class A:
> def __add__(self: T, other: T) -> T: ...
>
> There is a plan to support this in one of next releases of mypy.
>
> And now the reader should understand what is TypeVar, and you still have
the string there. Not that I'm saying it's a bad solution, but it fits pyi
files more than average-programmer-code.


> In general I think it is a bit early to judge what would be the best
> solution for forward references.
> (there are related issues like performance drop etc). More evidence is
> needed to decide a way forward.
>

The problem with waiting for more evidence is that more code will break if
the change require such breakage. At least side-effect in annotation
expressions should be "deprecated" and not guarantee when it happen, how
many times, etc.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Thu, Sep 22, 2016 at 11:02 PM David Mertz <me...@gnosis.cx> wrote:

> On Thu, Sep 22, 2016 at 12:59 PM, אלעזר <elaz...@gmail.com> wrote:
>
>> I don't If this feature is "nice, but does not worth the complication",
>>> then so be it; I can't claim I know better. I only speculate that it does
>>> not necessarily requires a new custom type.
>>>
>>
> I don't mean a runtime type here (necessarily), but rather a new type for
> the parser.  I.e. transform this actual expression into some sort of
> delayed expression when parsing.
>

Just as a demonstration, the parser can transform `EXP` into `lambda: EXP`
- and that's it. It will not solve everything (e.g. error messages and
.__annotation__ access as Alexander says), but it demonstrates the fact
that the change need not be so deep at all.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Thu, Sep 22, 2016 at 11:02 PM David Mertz <me...@gnosis.cx> wrote:

> On Thu, Sep 22, 2016 at 12:59 PM, אלעזר <elaz...@gmail.com> wrote:
>
>> I don't If this feature is "nice, but does not worth the complication",
>>> then so be it; I can't claim I know better. I only speculate that it does
>>> not necessarily requires a new custom type.
>>>
>>
> I don't mean a runtime type here (necessarily), but rather a new type for
> the parser.  I.e. transform this actual expression into some sort of
> delayed expression when parsing.
>
> I understood this, and that was my speculation. I will like an explanation
as to why is this necessary, but it's probably off topic.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Thu, Sep 22, 2016 at 10:58 PM David Mertz <me...@gnosis.cx> wrote:

> On Thu, Sep 22, 2016 at 12:35 PM, אלעזר <elaz...@gmail.com> wrote:
>
>> I think we're talking about different things here. I just referred to the
>> common need to use the name of the current class in type annotation
>>
>> class A:
>> def add(self, other: A) -> A: ...
>>
>
> Yeah, I find the need for using the string "A" here a wart. Rather than
> change the entire semantics of annotations, it feels like a placeholder for
> this meaning would be better.  E.g.:
>
> class A:
> def __add__(self, other: CLS) -> CLS: ...
>
> A static checker could do the magic of recognizing that special name
> easily enough (no harder than recognizing the quoted string).  At runtime
> 'CLS' could either just be a singleton with no other behavior... or perhaps
> it could be some sort of magic introspection object.  It's more verbose,
> but you can also spell it now as:
>
> class A:
> def __add__(self, other: type(self)) -> type(self): ...
>
> That's a little ugly, but it expresses the semantics we want.
>
> Mypy fails on this, and version 4.4 fails very hard and ugly, with errors
for each line to the end of the file + 1. It can be fixed of course, but
it's worth noting: mypy does not expect arbitrary expression - because it
is an annotation.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Thu, Sep 22, 2016 at 10:45 PM David Mertz <me...@gnosis.cx> wrote:

> On Thu, Sep 22, 2016 at 12:35 PM, אלעזר <elaz...@gmail.com> wrote:
>
>> In such a hypothetical future world we might come to allow, e.g.
>>> `Sequence[#CustomThing]` where some general lazy facility or indirection is
>>> indicated by the '#' (just a placeholder for this comment, not a
>>> proposal).  But if that comes about, it should be available everywhere, not
>>> only in annotations.
>>>
>>
>> I generally agree, but this future world must be very far and has many
>> consequences, whereas the story of annotations is special in that it's not
>> actually an expression, to the reader.
>>
>
> The CPython developers (of whom I'm not one, but I've followed them
> closely for 18 years) place a high value on simplicity in the parser and
> interpreter.  Adding a new custom type of thing that is an "annotation
> object" would be a special case with a high burden to show its utility.
>
> My feeling is that this burden is actually lower for a new "delayed eval
> object" that might conceivably be added at a syntax level.  In some sense,
> this would add just as much complexity as a new annotation object, but it
> would be something that applies many places and hence perhaps be worth the
> added complexity.
>
> I don't If this feature is "nice, but does not worth the complication",
then so be it; I can't claim I know better. I only speculate that it does
not necessarily requires a new custom type.

A delayed eval object will be very useful for initilizers, for the very
reason that the current behavior is surprising.

--
This made me think about Steven's argument above: it is not true that
expressions are evaluated when they are encountered, since

x = lambda: print(1)

prints nothing. So a colon before an expression hints about delayed
evaluation. This includes annotations and lambda.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Delay evaluation of annotations

[אלעזר]

On Thu, Sep 22, 2016 at 9:43 PM Steven D'Aprano <st...@pearwood.info> wrote:

> On Thu, Sep 22, 2016 at 05:19:12PM +0000, אלעזר wrote:
> > Hi all,
> >
> > Annotations of function parameters and variables are evaluated when
> > encountered.
>
> Right, like all other Python expressions in general, and specifically
> like function parameter default arguments.
>

Just because you call it "expression", when for most purposes it isn't - it
is an annotation. "Expression" is something that you need its value right
now, and "annotation" is something that, well, annotates the code you see
right now. Terminology is not the important thing, but that seems to be the
basis your argument here.

> This makes it necessary to use string representation for names
> > that are not yet bound, which affects almost every class definition.
>
> Almost every class? Really? I find that implausible. Still, I can see it
> affecting *many* class definitions, so let's not quibble.
>
> I retract this as a factual claim, I think we agree here.


> It is
> > also easy to forget, and the result might be a (very uninteresting)
> > exception in certain untested paths, e.g. inside functions.
>
> Unlikely, unless you're talking about functions nested inside other
> functions, or unusual (but legal and sometimes useful) conditional
> definitions:
>
I was thinking about the former, but yeah, uncovered code will fail at
runtime, possibly in production, for *no* real reason. I do not claim that
this is common, but it is definitely unnecessary - unlike initialization
expressions.

(which I would have liked to see delayed too, but I can understand the
reasons why this is strongly opposed; it *is* an expression. Sadly bugs are
*much* more common there).


> if condition:
> # forward reference to MyClass
> def f(arg:'MyClass'): ...
> else:
> # oops, untested path
> def f(arg:MyClass): ...
>
> class MyClass: ...
>
>
> But generally speaking, that sort of code is unusual, and besides, if
> you're doing this, either the static type checker won't be able to cope
> with it at all (in which case there's little point in annotating the
> function), or it will cope, and detect the invalid annotation.
>
> Why would it detect invalid annotation here? It shouldn't.

And of course the fact that I use annotated code does not necessarily mean
I also use type checkers.


> > Editors and
> > IDEs also don't handle it well; for example, PyDev does not consider
> > string-annotations as an occurrence of the name, and warns about unused
> > imports.
>
> I would call that a bug in PyDev.
>
> Agreed - given the current situation, which calls for bugs in IDEs and
requires unnecessary effort from any person or tool parsing the code.


> > I propose delaying evaluation of annotation-expressions by either keeping
> > the AST of the annotation, or turning it implicitly from EXP into
> "lambda:
> > EXP". Inspection code that is interested in this information can access
> it
> > be calling (or evaluating) it.
>
> -1 on complicating the simple Python model that expressions are
> evaluated when they are reached.
>
> Again, should not be called "expressions" but "annotations", and every
reader that understand types will not expect it to be evaluated. So the
current "simple" model is actually surprising. It's only easier on the
implementation side.

You would also complicate the introspection of annotations. With
> your proposal, *every* annotation would be a function, and every(?)
> inspection would require calling the function to find out what the real
> annotation is.
>

This argument was partially answered by Alexander before. Generally,
introspection *libraries* will be tiny bit more complicated. Introspection
user code will not. And people that write introspection code *must*
understand the nitty-gritty details of the language, whereas people that
read and write regular code need not.


> And what would that do to modules which use annotations for some other
> purpose? I know Guido is keen to discourage such alternative uses, but
> they're still legal, and a change like this would outright break them.
>

I don't know any concrete examples. Can you give any? Are these examples
use side effects on annotation evaluation? Lastly, do *you* consider it a
good idea, one that should be accounted for?


> Personally, I'm not convinced that it is a burden to expect people to
> remember to quote forward references. If they forget, they will nearly
> always get a NameError at runtime or a warning/error when they run the
> type checker.
>

It's just another irritating inconvenience making the write-test cycle
longer for no obvious reason (at l

[Python-ideas] Delay evaluation of annotations

[אלעזר]

Hi all,

Annotations of function parameters and variables are evaluated when
encountered. This makes it necessary to use string representation for names
that are not yet bound, which affects almost every class definition. It is
also easy to forget, and the result might be a (very uninteresting)
exception in certain untested paths, e.g. inside functions. Editors and
IDEs also don't handle it well; for example, PyDev does not consider
string-annotations as an occurrence of the name, and warns about unused
imports.

I propose delaying evaluation of annotation-expressions by either keeping
the AST of the annotation, or turning it implicitly from EXP into "lambda:
EXP". Inspection code that is interested in this information can access it
be calling (or evaluating) it.

It certainly isn't a backward compatible change, but I think it won't
affect too much code. On the positive side, it will make annotated code
much more pleasing to read, will be less surprising for beginners, and will
help editors in syntax highlighting and name lookup.

In short, it will shift the burden of handling type annotations from
standard code to inspection code, which is where I believe it should rest.

While this idea is quite obvious, I've found no clue in PEP-3107 as to why
will it be a bad one (no such "rejected proposal" there), so I raise the
question here.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

This class should be put somewhere. If you need it from inside a method,
you have to choose where to put it. If it is inside the method, it forces
the reader to scan it thoroughly to verify there's no "real" closure there,
and it also makes your methods significantly longer. If it is outside the
whole class, it might be very far away from your logic, doing a very simple
thing that is not needed anywhere else.

Elazar

On Wed, Sep 21, 2016 at 5:32 AM Dan Sommers <d...@tombstonezero.net> wrote:

> On Tue, 20 Sep 2016 15:29:36 +, אלעזר wrote:
>
> > The alternative to partial is writing a closure in the form of a
> > function, that needs to be carefully inspected to verify that it is
> > indeed just a partial application and not something more complex. It
> > has more opportunity for introducing an error. And it's longer and
> > adds distance between related parts of the code.
>
> While I'm not usually one to promote object oriented programming,
> another Python alternative is a class with a __call__ method; e.g.:
>
> class Adder:
> def __init__(self, addend_one):
> self.addend_one = addend_one
> def __call__(self, addend_two):
> return self.addend_one + addend_two
>
> add_5 = Adder(5)
> print(add_5(4))
>
> I like the way the whole thing is bundled up into a class.  Yes, it's
> more verbose than a lambda expression or a partial function application,
> but I find it very readable and its intent is usually pretty obvious.
>
> Instances are closures in disguise (and they're all just different ways
> of hiding state of one kind or another).
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

On Wed, Sep 21, 2016 at 12:04 AM Terry Reedy  wrote:

> On 9/20/2016 11:51 AM, Guido van Rossum wrote:
>
> ... (The greater flexibility of lambda, pointed out by David Mertz, is
> another.)
>
>
I just wanted to point out that the greater flexibility of lambda is a very
good reason *not* to choose it: more flexibility implies being harder to
reason about. I recommend reading this article about "Principle of Least
Power"
http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

On Tue, Sep 20, 2016 at 9:18 PM Stephan Houben  wrote:

> I must admit I am a bit partial to partial, you can do fun things like
> this:
>
> >>> from functools import partial
> >>> @partial(partial, partial)
> ... def add(x, y):
> ... return x+y
> ...
> >>> add(3)(4)
> 7
>
> I suppose that isn't exactly going to convince Guide to put it in
> builtins, though.
>
> I quietly LOLed, but note that for three arguments you'll need

@partial(partial, partial(partial, partial))
def add(a, b, c): return a + b + c

>>> add(1)(2)(3)
6

So @curry is the transitive closure or something :)

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

Guido, can you please elaborate?

"What's going on" is usually that the same arguments are going to be passed
over and over again, and the programmer wanted to avoid this repetition.
The other option is adjusting the function to a predefined interface.

The alternative to partial is writing a closure in the form of a function,
that needs to be carefully inspected to verify that it is indeed just a
partial application and not something more complex. It has more opportunity
for introducing an error. And it's longer and adds distance between related
parts of the code.

Elazar

בתאריך יום ג׳, 20 בספט' 2016, 18:20, מאת Guido van Rossum ‏<
gvanros...@gmail.com>:

> I am radically opposed to this proposal. Every time I see a partial
> application I wonder endlessly about what's going on.
>
> --Guido (mobile)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Moreover, being able to do it programmatically is a security risk, since it
requires elevated privileges that I don't know how to drop, and most people
will not think about doing, but a library implementation will.

So if someone uses subprocess.run(), and the system asks the user for
elevated privileges, a bug in later code can easily cause serious harm
instead of failing. Yes, untrusted code should be sandboxed - but it isn't,
more often than not.

Elazar

‪On Tue, Sep 20, 2016 at 2:02 PM ‫אלעזר‬‎ <elaz...@gmail.com> wrote:‬

> I think I generally understand concerns, and partially agree. I'm
> certainly not dismissing them. I only try to understand what are the
> precise problems and why the current situation - with dangerous functions
> at reach, easily buried deep in the code instead of marked on the top of
> the script - is so much better.
>
> Elazar
>
> On Tue, Sep 20, 2016 at 1:56 PM Paul Moore <p.f.mo...@gmail.com> wrote:
>
>> On 20 September 2016 at 11:46, אלעזר <elaz...@gmail.com> wrote:
>> > So it should be something like
>> >
>> > from unsafe.__pip__ import benchmark
>> >
>> > Where unsafe is the hypothetical namespace in which exec(), eval() and
>> > subprocess.run() would have reside given your concerns.
>>
>> In my opinion, it should be
>>
>> # Please install benchmark using pip to run this script
>>
>> Or you should run the script using a dedicated runner like rwt. Or you
>> can depend on a custom import hook that makes "from __pip__
>> install..." work as you want. I'm just saying that I don't want core
>> Python to implicitly install packages for me. But that's simply a
>> personal opinion. I'm not trying to persuade you you're wrong, just
>> trying to explain my position. We can agree to differ. It certainly
>> doesn't seem to me that there's any need for you to modify your
>> proposal to suit me, it's unlikely I'll like any variation you're
>> going to be happy with, which is fine (you're under no obligation to
>> convince me).
>>
>> Paul
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

I think I generally understand concerns, and partially agree. I'm certainly
not dismissing them. I only try to understand what are the precise problems
and why the current situation - with dangerous functions at reach, easily
buried deep in the code instead of marked on the top of the script - is so
much better.

Elazar

On Tue, Sep 20, 2016 at 1:56 PM Paul Moore <p.f.mo...@gmail.com> wrote:

> On 20 September 2016 at 11:46, אלעזר <elaz...@gmail.com> wrote:
> > So it should be something like
> >
> > from unsafe.__pip__ import benchmark
> >
> > Where unsafe is the hypothetical namespace in which exec(), eval() and
> > subprocess.run() would have reside given your concerns.
>
> In my opinion, it should be
>
> # Please install benchmark using pip to run this script
>
> Or you should run the script using a dedicated runner like rwt. Or you
> can depend on a custom import hook that makes "from __pip__
> install..." work as you want. I'm just saying that I don't want core
> Python to implicitly install packages for me. But that's simply a
> personal opinion. I'm not trying to persuade you you're wrong, just
> trying to explain my position. We can agree to differ. It certainly
> doesn't seem to me that there's any need for you to modify your
> proposal to suit me, it's unlikely I'll like any variation you're
> going to be happy with, which is fine (you're under no obligation to
> convince me).
>
> Paul
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

On Tue, Sep 20, 2016 at 1:42 PM Paul Moore <p.f.mo...@gmail.com> wrote:

> On 20 September 2016 at 00:28, אלעזר <elaz...@gmail.com> wrote:
> > On Tue, Sep 20, 2016 at 2:20 AM Stephen J. Turnbull
> > <turnbull.stephen...@u.tsukuba.ac.jp> wrote:
> >>
> >> אלעזר writes:
> >>
> >>  > Another use case, though I admit not the top priority of anyone here,
> >> is
> >>  > that of assignment checkers. In most courses I took at the
> university,
> >> the
> >>  > person who checks the assignments says something like "you are
> allowed
> >> to
> >>  > use only this this and this libraries", in order not to mess with
> >> unknown
> >>  > dependencies from tens of students (I am talking about advanced
> >> courses,
> >>  > where the method I use to solve the problem is unimportant or only
> >> requires
> >>  > explanation). With this statement they can simply state "you can
> import
> >>  > pip".
> >>
> >> In other words, you're advocating a feature that allows script writers
> >> to download, install, and execute arbitrary, unsandboxed code on any
> >> machine where the script is run.  That sounds ... *scary*, when put
> >> that way.  Remember, you're advocating this on behalf of people who by
> >> assumption are infants years below the age of consent.
> >>
> > Let me understand. Your argument is "installing pip modules is unsafe,
> and
> > therefore we should make it less usable, where the appropriate amount of
> > (un)usability is running cmd and then `pip install unsafe`" ?
>
> The argument is that if someone posts a script that says it does
> something innocuous, for example "benchmark showing that X is faster
> than Y", people will scan it, see that it looks OK, and run it. They
> have a reasonable expectation that it's not a security risk.
>
> If it requires a benchmarking module from PyPI, they may not
> immediately notice that "from __pypi__ import benchmark" opens up a
> security risk. On the other hand, being explicitly told to run a
> command whose sole purpose is to download and install an external tool
> clearly indicates to them that they need to be aware of what's
> happening. Likely they will simply do so and it's no big deal. But in
> certain environments they may have to pause and possibly even check
> with their security team as to whether that tool has been approved.
>
> It's not about "making it less usable", it's about ensuring that the
> implications are clear - explicit is better than implicit, in effect.
> Which is a particularly important principle when security risks such
> as "downloading arbitrary code from the internet" is involved.
>
>
So it should be something like

from unsafe.__pip__ import benchmark

Where unsafe is the hypothetical namespace in which exec(), eval() and
subprocess.run() would have reside given your concerns.

 Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

I believe that at least some of these problems can be addressed given that
pip *knows* that this import is an in-script import. So the list of corner
cases will be shorter.

Elazar

On Tue, Sep 20, 2016 at 1:35 PM Paul Moore <p.f.mo...@gmail.com> wrote:

> On 19 September 2016 at 23:46, אלעזר <elaz...@gmail.com> wrote:
> >> import pip
> >> pip.install('attrs')
> >> import attr
> >
> > Please forgive me for my ignorance, but it doesn't work as written -
> what's
> > the actual method?
>
> As David Mertz said, pip.main(['install', 'attrs']) works right now,
> but it is NOT a supported use of pip[1]. To be 100% explicit, the only
> supported way of doing this is
>
> import sys
> from subprocess import run
> run([sys.executable, '-m', 'pip', 'install', 'attrs'])
>
> I suggested a hypothetical "pip.install" method as there is currently
> some discussion on the pip tracker about providing a supported install
> method. But it doesn't exist yet. Sorry for being confusing.
>
> While on the whole subject of this, I should also point out that there
> are a lot of potential issues with installing new packages while a
> Python program is running. They are all low-probability, and easy to
> avoid if you're not doing weird things, but for a generally-promoted
> mechanism, we need to explain the corner cases[2], and an approach
> with a list of caveats longer than the main documentation is
> problematic.
>
> 1. If the install fails, you need to catch that and report it to the
> user, in a more friendly manner than pip's output. For example if the
> user has no C compiler and you need a C extension to be built.
> 2. You quite possibly want to suppress pip's output if it's *not* a
> failure, as it'll clutter up the program's real output.
> 3. If the code has already imported foo.bar, then you install a new
> version of foo (there are discussions as to whether pip install foo
> should automatically imply --upgrade, so even if it won't do that by
> default now, it might in the future), and maybe that new version
> doesn't have a bar submodule. So now you have a weird mix of old and
> new code in your process.
> 4. The install mechanism sometimes (I can't recall the details) caches
> the fact that it couldn't import a module. If it does that and then
> later you pip install that module, imports will still fail because the
> information is cached.
>
> I'm still not at all clear why any of this is so much better than a
> comment at the top of the script
>
> # To run this script, you need to "pip install attrs" first
>
> Paul.
>
> [1] We've had people report issues where pip breaks their logging
> config, for example, because pip uses logging but doesn't expect to be
> run from user code that also does so.
> [2] That "run([sys.executable, ...])" invocation doesn't work in an
> embedded program, for example, where sys.executable isn't "python".
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

Yeah I did say it was a strawman :)

On Tue, Sep 20, 2016 at 11:17 AM Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Sep 20, 2016 at 6:09 PM, אלעזר <elaz...@gmail.com> wrote:
> > I meant something like making it a "__bind__" (just a strawman
> suggestion)
> > and do the same lookup as foo() does, and using a (wrong)
> > functional-programming-inspired syntax
> >
> > foo 5 ()
> >
> > Such a syntax will have the side benefit of allowing calling print in a
> > similar way to Python2, which people seem to love.
> >
> > print "hello" ()
> >
>
> Python has a rule that syntax shouldn't look like grit on Tim's
> screen. In this case, it looks like the *absence of* grit, which is
> even worse :) You're giving meaning to the abuttal of two tokens, the
> first of which must be callable but the second can be anything. And it
> creates the scary situation of giving valid-but-useless meaning to
> something all too common in Py2 code:
>
> print "hello"
>
> This would now create a function that, if called, would print "hello",
> but then abandons it without a second thought. So it'd work in 2.7,
> fail with an opaque error in 3.3, fail with a more informative error
> in 3.5, and silently do nothing in 3.7. No thank you! :)
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

On Tue, Sep 20, 2016 at 10:54 AM Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Sep 20, 2016 at 5:01 PM, אלעזר <elaz...@gmail.com> wrote:
> > But the foo() finds the function to call, so foo.bind() could be made to
> > find it too.
>
> class Demo:
> def __init__(self):
> self.bind = 42
> def __call__(self):
> print("I got called!")
>
> foo = Demo()
>
> You can most certainly call foo(), but foo.bind() will bite you.
>
> With a stand-alone function bind(foo), it can use protocols like __call__.
>
> I meant something like making it a "__bind__" (just a strawman suggestion)
and do the same lookup as foo() does, and using a (wrong)
functional-programming-inspired syntax

foo 5 ()

Such a syntax will have the side benefit of allowing calling print in a
similar way to Python2, which people seem to love.

print "hello" ()

This strawman proposal has many downsides I guess. My point being, this can
be made to work, but it's probably not worth it.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Clear screen command for the REPL

[אלעזר]

On Tue, Sep 20, 2016 at 4:56 AM Steven D'Aprano  wrote:

> On Mon, Sep 19, 2016 at 01:35:53PM -0700, João Matos wrote:
> > Hello,
> >
> > I don't see why creating a clear command would interfere with
> dict.clear()
> > which is a function/method.
>
> For the same reason that you can't have a method called foo.while or
> foo.if or foo.raise. If clear is a "command" (a statement) it would need
> to be a keyword, like while, if and raise.
>
>
> This is since in Python there are no contextual keywords (like "override"
and "final" in C++). I remember encountering error in a Django project
where accessing u.pass was a syntax error, but there *was* a field "pass"
in u and they had to resort to getattr(u, "pass").
What is the reasoning behind that decision?

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Make partial a built-in

[אלעזר]

But the foo() finds the function to call, so foo.bind() could be made to
find it too.

בתאריך יום ג׳, 20 בספט' 2016, 08:24, מאת Stefan Behnel ‏<stefan...@behnel.de
>:

> אלעזר schrieb am 19.09.2016 um 17:59:
> > If at all, it should be function.bind(). It was discussed and dropped; I
> > don't remember why, but one problem is that it looks like an in-place
> > modification.
>
> IMHO, the main drawback of that solution is that it only works for
> functions and not for any other callable in Python. It would introduce an
> inconsistency without solving the problem in general.
>
> Stefan
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

On Tue, Sep 20, 2016 at 3:06 AM Ethan Furman <et...@stoneleaf.us> wrote:

> On 09/19/2016 04:38 PM, אלעזר wrote:
>
> > I was talking specifically about advanced courses, in which an assignment
> >  is "implement a side-channel attack using data" and you can use whatever
> >  library you like.
>
> Am I misunderstanding, or did you just say you want this new functionality
> in order to implement attacks on people's computers?
>

This is completely off topic (the feature is irrelevant to the contents of
the course).
But - you are misunderstanding :) And yes I took a course in which I was
expected to demonstrate attacks on people's computers using e.g.
correlation power analysis. It was based on fake data but yes. They also
give courses in which you learn that there's these things called man in the
middle, buffer overflows and return-oriented programming!

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

But grepping and piping could work I assume?

On Tue, Sep 20, 2016 at 2:41 AM Chris Angelico  wrote:

> On Tue, Sep 20, 2016 at 9:27 AM, Xavier Combelle
>  wrote:
> > I find the idea of tracking the dependencies in the script might be a
> > good idea.
> > However, magically downloading without warning the user is in my point
> > of view for sure a bad idea.
> > I would far prefer that pip could scan a script to know the dependencies.
> > (A little bit like a requirements.txt but inside the script)
> > A special comment or docstring would do the job. for example
> >
> > """
> > pip_requirements:
> > - requests >0.0
> > - asyncio
> > 
> >
> > to run the script it would be at the first time a  two step process
> > for example:
> >
> > python3 -m pip --script-dependencies [--user] my_script.py
> > python3 my_script.py
>
> How about this:
>
> python3 -m pip install -r requirements.txt
> python3 my_script.py
>
> That already works, but the requirements have to be in a separate text
> file, not in the .py file itself. If you want a docstring-based
> solution, I'd piggy-back it on requirements.txt, and use the same
> format (eg if the docstring starts "pip_requirements:", the rest of it
> is a requirements.txt file). Downside: Docstrings can be easily parsed
> and read *IF* you can successfully import the module. Which you can't,
> if you still need all those dependencies. So there would have to be
> some kind of import hook that crafts a dummy package with an infinite
> number of dummy subpackages, to allow all forms of import to trivially
> succeed - "import X", "import X.Y", "from X.Y import Z", etc. This may
> well be more hassle than it's worth.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

On Tue, Sep 20, 2016 at 2:34 AM Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Sep 20, 2016 at 9:20 AM, Stephen J. Turnbull
> <turnbull.stephen...@u.tsukuba.ac.jp> wrote:
> > אלעזר writes:
> >
> >  > Another use case, though I admit not the top priority of anyone here,
> is
> >  > that of assignment checkers. In most courses I took at the
> university, the
> >  > person who checks the assignments says something like "you are
> allowed to
> >  > use only this this and this libraries", in order not to mess with
> unknown
> >  > dependencies from tens of students (I am talking about advanced
> courses,
> >  > where the method I use to solve the problem is unimportant or only
> requires
> >  > explanation). With this statement they can simply state "you can
> import
> >  > pip".
> It's more than that, though. When a student is given an assignment,
> it's usually to help said student to learn to *write code*, not to
> learn how to walk into the plumbing store of PyPI and look for
> something that approximates to the job being done. Maybe it's
> different at university, but with my students, it's always been "no
> external libraries" (and in some cases, a common-sense avoidance of
> obvious solutions from the standard library - if you teach someone how
> to implement a sort and the response is simply "lst.sort()", it's not
> exactly implementing anything). So the use-case for this isn't nearly
> as big as it might be. By the time you get to writing large
> applications (again, taking it from my students' work: a Flask-based
> web app), it's not illogical to have a requirements.txt and standard
> pip incantations.
>
> ChrisA
>

I was talking specifically about advanced courses, in which an assignment
is "implement a side-channel attack using data" and you can use whatever
library you like.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Xavier, how is connecting and installing different from a windows popup
"this software requires changes to your firewall settings" or "requires
elevated privileges" which already happens. I am all for a two-step
process, but I think it should be more user friendly, and it can be done as
a Python command from inside the script.

If I send you a small script, it should be treated in the same way as if I
send you a program - an installer - not as a code that you should
incorporate into your already existing code base.

‪On Tue, Sep 20, 2016 at 2:28 AM ‫אלעזר‬‎ <elaz...@gmail.com> wrote:‬

> On Tue, Sep 20, 2016 at 2:20 AM Stephen J. Turnbull <
> turnbull.stephen...@u.tsukuba.ac.jp> wrote:
>
>> אלעזר writes:
>>
>>  > Another use case, though I admit not the top priority of anyone here,
>> is
>>  > that of assignment checkers. In most courses I took at the university,
>> the
>>  > person who checks the assignments says something like "you are allowed
>> to
>>  > use only this this and this libraries", in order not to mess with
>> unknown
>>  > dependencies from tens of students (I am talking about advanced
>> courses,
>>  > where the method I use to solve the problem is unimportant or only
>> requires
>>  > explanation). With this statement they can simply state "you can import
>>  > pip".
>>
>> In other words, you're advocating a feature that allows script writers
>> to download, install, and execute arbitrary, unsandboxed code on any
>> machine where the script is run.  That sounds ... *scary*, when put
>> that way.  Remember, you're advocating this on behalf of people who
>> by assumption are infants years below the age of consent.
>>
>> Let me understand. Your argument is "installing pip modules is unsafe,
> and therefore we should make it less usable, where the appropriate amount
> of (un)usability is running cmd and then `pip install unsafe`" ?
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

On Tue, Sep 20, 2016 at 2:20 AM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> אלעזר writes:
>
>  > Another use case, though I admit not the top priority of anyone here, is
>  > that of assignment checkers. In most courses I took at the university,
> the
>  > person who checks the assignments says something like "you are allowed
> to
>  > use only this this and this libraries", in order not to mess with
> unknown
>  > dependencies from tens of students (I am talking about advanced courses,
>  > where the method I use to solve the problem is unimportant or only
> requires
>  > explanation). With this statement they can simply state "you can import
>  > pip".
>
> In other words, you're advocating a feature that allows script writers
> to download, install, and execute arbitrary, unsandboxed code on any
> machine where the script is run.  That sounds ... *scary*, when put
> that way.  Remember, you're advocating this on behalf of people who
> by assumption are infants years below the age of consent.
>
> Let me understand. Your argument is "installing pip modules is unsafe, and
therefore we should make it less usable, where the appropriate amount of
(un)usability is running cmd and then `pip install unsafe`" ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Sounds good, thanks.

So I'll suggest adding the method mentioned in Paul's code as a promised
API, either as pip.install or otherwise. `pip.main(['install', 'boltons'])` is
not the clearest it can get.

Elazar

On Tue, Sep 20, 2016 at 2:02 AM David Mertz <me...@gnosis.cx> wrote:

> >>> import pip
> >>> pip.main(['install', 'boltons'])
> Requirement already satisfied (use --upgrade to upgrade): boltons in
> ./anaconda/lib/python3.5/site-packages
> 0
>
> This is easy, short, and uses existing syntax and an existing module.
> Yes, I know technically the API isn't promised; I wouldn't object to making
> that API official.  But PLEASE, no new syntax for doing this!
>
> Btw. On failure:
>
> >>> pip.main(['install', 'nonesuch'])
> Collecting nonesuch
>   Could not find a version that satisfies the requirement nonesuch (from
> versions: )
> No matching distribution found for nonesuch
> 1
>
>
> On Mon, Sep 19, 2016 at 3:46 PM, אלעזר <elaz...@gmail.com> wrote:
>
>> On Tue, Sep 20, 2016 at 1:40 AM Paul Moore <p.f.mo...@gmail.com> wrote:
>>
>>> On 19 September 2016 at 23:13, אלעזר <elaz...@gmail.com> wrote:
>>> > Alternative syntax might be that of import decorators
>>> >
>>> > @from_pip("attrs")
>>> > import attr
>>>
>>> At which point you may as well just (insert disclaimer about "using
>>> pip's internal API isn't supported" here, and glossing over the fact
>>> that pip doesn't yet have an install function with this API) do
>>>
>>> import pip
>>> pip.install('attrs')
>>> import attr
>>>
>>
>> Please forgive me for my ignorance, but it doesn't work as written -
>> what's the actual method?
>>
>> Elazar
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Paul, I understand what you say, except the part of compatibility with
Python3.5. Of course such a change is targeting future version more than
the current and previous versions. If we have this syntax for Python3.6,
users of Python3.9 will find many copy-pastable answers written this way,
and other answers will have comments saying "just add this statement to the
beginning of your script".

On Mon, Sep 19, 2016 at 11:24 PM Paul Moore <p.f.mo...@gmail.com> wrote:

> On 19 September 2016 at 19:52, אלעזר <elaz...@gmail.com> wrote:
> > Of course it doesn't  address the issues - for example, running a python
> > script from the file manager will result in a failed execution,
> unexplained.
>
> What you're talking about here is deployment of Python "applications"
> (where in your case, you're focusing on single-file scripts, but the
> problem is broader). Beyond a certain point, as I'm sure you realise,
> you can't solve this issue - someone tries to run a Unix-only script
> on Windows, for example. So really, we're looking at ways of improving
> the current state of affairs. As things stand, deploying "standalone
> applications" in Python is not particularly easy, with dependency
> management being only one of the issues.
>
> Some options for deployment come to mind - I'm sure none of them seem
> quite as easy as "just run this script", but you should remember that
> even "just run this script" doesn't always work as well as you'd like
> - on Unix you need to make the script executable and have a shebang
> line, on Windows the file extension needs to be right and registered.
> Not all users have the right (or sometimes any) Python version on
> their PATH, etc. So even "just run this script" isn't always simple
> even if the script has no dependencies outside the stdlib.
>
> So, some options:
>
> 1. The previously-mentioned rwt makes "just run this script" work as
> you want, as long as you have rwt installed. Conceded, it isn't
> available by default, but it's not hard to install as a one-off task.
> And there's a discussion ongoing about including it with pip as "pip
> run" so "pip run myscript.py" (or "py -m pip run myscript.py" on
> Windows - remember what I said about PATH? ;-)) will run a script with
> its dependencies.
> 2. You can bundle your script with its dependencies using zipapp. On
> Unix, you can even make the resulting file executable, and on Windows,
> the .pyz extension is registered for this.
> 3. Someone could write a tool to scan a script for a
> specially-formatted comment at the top that described its
> dependencies, and installed them. That (in effect) adds an "install"
> step before the script can be run, but lots of utilities have that, so
> it's not completely unheard of. And if it's GUI users you're concerned
> about, you could make that a GUI utility. Drag the script onto it, it
> lists the dependencies and confirms you want them installed, then
> offers to run the script.
> 4. There are existing platform-specific options like py2exe or
> cx_Freeze. Probably a bit heavyweight for a "simple script", though.
>
> Of course, none of these options cover the use case of "I saw this
> nice script on Stack Overflow, so I copied it, and now I want to run
> it on my PC". But honestly, if it has some dependencies, and the
> script author didn't say "you need to install FOO and BAR", or the
> user doesn't know how to install those dependencies, you're probably
> out of luck anyway. Even if the proposed "from __pypi__ import foo"
> syntax existed, I doubt many scripts posted on the web would use it -
> if only because then, Python 3.5 users wouldn't be able to run the
> scripts!
>
> IMO, for command line use, rwt is 99% of the way to what you want. For
> GUI use, there's nothing specific to this problem right now, but
> someone could certainly write something. A syntax change to the
> language would likely be *less* useful, as people would not be able to
> use it unless they were deliberately only targeting Python 3.7 and
> later, which seems optimistic at best...
>
> Paul
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Wow chris, that's a nice one!

Of course it doesn't  address the issues - for example, running a python
script from the file manager will result in a failed execution, unexplained.

On Mon, Sep 19, 2016 at 9:48 PM Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Sep 20, 2016 at 4:43 AM, MRAB <pyt...@mrabarnett.plus.com> wrote:
> > On 2016-09-19 18:20, אלעזר wrote:
> >>
> >> Obviously
> >>
> >> from __pip__ import "run-lambda>=0.1.0"
> >>
> >> Which is ugly but not my fault :)
> >>
> > [snip]
> > One possible problem I can see is that if it's quoted you might think
> that
> > it's an expression and that you could also write:
> >
> > package = "run-lambda>=0.1.0"
> > from __pip__ import package
>
> One surprisingly effective option is this:
>
> import blargle.spam # Solution: pip install blarg
>
> Comes out looking like this:
>
> Traceback (most recent call last):
>   File "useblarg.py", line 1, in 
> import blargle.spam # Solution: pip install blarg
> ModuleNotFoundError: No module named 'blargle'
>
> (in older Pythons, that says "ImportError" rather than the more
> specific subclass, but same diff)
>
> You can put whatever you like onto that command line.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Ryan I assume any error could be translated to your version through the
`from ` statement too?

Regarding the rabbit hole, the internet connection and privileges were
required for running this script in the first place. I only suggest giving
the developer - perhaps not a software engineer at all - an easy way not to
require the (possibly inexperienced) user to do it manually.

Put it another way, you a copy-pasted code from SO is not needed to use
only stdlib now.

On Mon, Sep 19, 2016 at 8:34 PM Ryan Gonzalez <rym...@gmail.com> wrote:

> This sounds like a great idea, but I've seen stuff like this done before,
> and it never ends well. You end up with a gargantuan-sized rabbit hole,
> since running a basic script could now involve using an internet connection
> and potentially root permissions.
>
> IF one were to go this route, I prefer bundling a `requirements.txt` into
> the distribution and then doing something like this in the script:
>
> import pkgcheck
> pkgcheck.check_requirements('requirements.txt')
>
> If any of the requirements were missing, then something like this would
> happen:
>
> Traceback (most recent call last):
> ...
> MissingPackageError: This script requires package(s) which are not
> installed: mypackage>=1.0, other_package
>
> That way, you don't get the weird import errors, but you don't have to
> worry about all the subtleties of automatic downloading.
>
> --
> Ryan
> [ERROR]: Your autotools build scripts are 200 lines longer than your
> program. Something’s wrong.
> http://kirbyfan64.github.io/
> On Sep 19, 2016 11:26 AM, "אלעזר" <elaz...@gmail.com> wrote:
>
>> Many proposals to add something to stdlib are rejected here with the
>> suggestion to add such library to pypi first. As noted by someone, pypi is
>> not as reachable as stdlib, and one should install that package first,
>> which many people don't know how. Additionally, there is no natural
>> distinction between 3rd party dependencies and in-project imports (at least
>> in tiny projects).
>>
>> This can be made easier if the first line of the program will declare the
>> required library, and executing it will try to download and install that
>> library if it is not installed yet. Additionally, the 3rd party
>> dependencies will be more explicit, and editors can then allow you to
>> search for them as you type.
>>
>> Of course it is *not* an alternative for real dependency management, but
>> it will ease the burden on small scripts and tiny projects - which today
>> simply break with errors that many users does not understand, instead of
>> simply asking permission to install the dependency.
>>
>> Elazar
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Obviously

from __pip__ import "run-lambda>=0.1.0"

Which is ugly but not my fault :)

On Mon, Sep 19, 2016 at 8:16 PM Sven R. Kunze <srku...@mail.de> wrote:

> I can definitely understand your point.
>
> The only issue with it (besides that it doesn't seem to be a good way for
> dependency management) is how do you manage the syntax involved here?
>
> Pip provides distributions. Each distribution contains a set of packages
> and modules. The latter can be imported, the former not. That's also due to
> the fact that the name of distribution can contain minuses:
>
>
> from __pip__ import nova-lxd   # would this work?
>
> What about versions?
>
> from __pip__ import run-lambda>=0.1.0   # would this work?
>
>
> Maybe, I thinking too complicated here but if it works for, say,
> "requests" people tend to want it for special cases as well. :)
>
>
> Cheers,
>
> Sven
>
> On 19.09.2016 18:55, אלעזר wrote:
>
> A library in PyPi  still requires installing it, which undermine many of
> the benefits. It won't help me with my gist/activestate recipe, code that I
> send to a friend, etc. I want to lower the barrier of inexperienced users.
>
> As a documentation of dependencies it will suffice indeed.
>
> Elazar
>
> On Mon, Sep 19, 2016 at 7:38 PM Ethan Furman <et...@stoneleaf.us> wrote:
>
>> On 09/19/2016 09:25 AM, אלעזר wrote:
>>
>> > Many proposals to add something to stdlib are rejected here with the
>> suggestion to add such library to pypi first. As noted by someone, pypi is
>> not as reachable as stdlib, and one should install that package first,
>> which many people don't know how. Additionally, there is no natural
>> distinction between 3rd party dependencies and in-project imports (at least
>> in tiny projects).
>> >
>> > This can be made easier if the first line of the program will declare
>> the required library, and executing it will try to download and install
>> that library if it is not installed yet. Additionally, the 3rd party
>> dependencies will be more explicit, and editors can then allow you to
>> search for them as you type.
>> >
>> > Of course it is *not* an alternative for real dependency management,
>> but it will ease the burden on small scripts and tiny projects - which
>> today simply break with errors that many users does not understand, instead
>> of simply asking permission to install the dependency.
>>
>> This should start out as a library on PyPI.  (Sorry, couldn't resist. ;)
>>
>> Actually, it should.  Perhaps a name of "import_pip" would make sense?
>> Any hurdles faced by this library would be (mostly) the same as a stdlib
>> version.
>>
>> --
>> ~Ethan~
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> ___
> Python-ideas mailing 
> listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] from __pip__ import

[אלעזר]

Thanks Joonas. I withdraw my proposal - nothing more is strictly needed. It
should be idiomatic somehow, but I don't have any specific suggestion.

On Mon, Sep 19, 2016 at 7:59 PM Joonas Liik <liik.joo...@gmail.com> wrote:

> On 19 September 2016 at 19:55, אלעזר <elaz...@gmail.com> wrote:
> > A library in PyPi  still requires installing it, which undermine many of
> the
> > benefits. It won't help me with my gist/activestate recipe, code that I
> send
> > to a friend, etc. I want to lower the barrier of inexperienced users.
> >
> > As a documentation of dependencies it will suffice indeed.
> >
> > Elazar
> >
> > On Mon, Sep 19, 2016 at 7:38 PM Ethan Furman <et...@stoneleaf.us> wrote:
> >>
> >> On 09/19/2016 09:25 AM, אלעזר wrote:
> >>
> >> > Many proposals to add something to stdlib are rejected here with the
> >> > suggestion to add such library to pypi first. As noted by someone,
> pypi is
> >> > not as reachable as stdlib, and one should install that package
> first, which
> >> > many people don't know how. Additionally, there is no natural
> distinction
> >> > between 3rd party dependencies and in-project imports (at least in
> tiny
> >> > projects).
> >> >
> >> > This can be made easier if the first line of the program will declare
> >> > the required library, and executing it will try to download and
> install that
> >> > library if it is not installed yet. Additionally, the 3rd party
> dependencies
> >> > will be more explicit, and editors can then allow you to search for
> them as
> >> > you type.
> >> >
> >> > Of course it is *not* an alternative for real dependency management,
> but
> >> > it will ease the burden on small scripts and tiny projects - which
> today
> >> > simply break with errors that many users does not understand, instead
> of
> >> > simply asking permission to install the dependency.
> >>
> >> This should start out as a library on PyPI.  (Sorry, couldn't resist. ;)
> >>
> >> Actually, it should.  Perhaps a name of "import_pip" would make sense?
> >> Any hurdles faced by this library would be (mostly) the same as a stdlib
> >> version.
> >>
> >> --
> >> ~Ethan~
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> using pip from within python is not that difficult already.
> as can be seen with a glance to:
>
> http://stackoverflow.com/questions/12332975/installing-python-module-within-code
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] typing.modifiers

[אלעזר]

Thank you all!

אלעזר
(AKA Elazar)

On Sat, Sep 17, 2016 at 4:53 AM Steven D'Aprano <st...@pearwood.info> wrote:

> On Sat, Sep 17, 2016 at 03:39:08AM +1000, Chris Angelico wrote:
> > On Fri, Sep 16, 2016 at 11:22 PM, אלעזר <elaz...@gmail.com> wrote:
> > > P.S. how do I change the name in my quotes? I believe אלעזר is not
> very easy
> > > to address...
> > >
> >
> > TBH I wouldn't worry about it. If people can't cite names using Hebrew
> > script, that's their problem, not yours. :)
>
> [in-joke, that Chris will get]
> But how do we know that אלעזר is his real name? It looks made up to me.
> [/in-joke]
>
> Still, thank you Elazar for also providing a Latin-1 compatible name
> which is readable and pronounceable by English speakers.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] typing.modifiers

[אלעזר]

On Fri, Sep 16, 2016 at 1:16 PM Steven D'Aprano <st...@pearwood.info> wrote:

> On Fri, Sep 16, 2016 at 12:10:22AM +0000, אלעזר wrote:
>
> [...]
> > Benefits of putting such a collection in stdlib (instead of as an
> external
> > package) include:
>
> Slow down! Before getting all excited about adding these typing hints(?)
> into typing.modifiers, you first have to convince people that they are
> useful and deserve a place in the std library.
>
> What problem are these hints/classes supposed to solve? What
> solutions already exist? Why aren't those solutions good enough?
>


> It is partially described below: mainly, reasoning and readability. I will
try to elaborate, and please forgive me if what I'm writing is obvious to
everyone here. Note that by "readability" I mean "conventional,
non-repetitive syntax, the allow human reasoning", not the subjective issue
(which is also important).

Q. Why standard?

A. Python is not very good at allowing strict reasoning. The confidence in
Python programs, insofar it exists, comes from its reliance on very strong
conventions ("One Way To Do It") and mostly-explicit control flow and name
binding. Data flow, however, is very much implicit since the language
encourages mutation.

Tools that employ conventions are generally linters; these are very
useful in finding trivial bugs, but have zero guarantees about their
absence - unless the inference is trivial. linters and IDEs can also use
basic form of type information, but will not analyze it deeply; hence the
need for convention for modifiers that themselves constrain mutability,
subclassing, etc.As a gradually-typed language, constraints are opt-in at
best, but there should be a way to opt-them-in in a conventional way. Type
hints are very good, and the suggested modifiers complement them.

---
Q. What solutions already exist? Why aren't those solutions good enough?

A:
* Named: There are no conventions and no standard way to prohibit
monkey-patching, since this flexibility is often useful. That's fine. But
what if I want to tell the other programmer that this class should not be
monkey patched, but its fields are mutable? Currently the general solution
is something like

class Person:
name = None
id = None
def __init__(self, name, id):
self.name = name
self.od = id

Say we ignore the repetition. Will tools catch the last typo? Some might
warn, but how should a tool know that it is not intentional? There's no
standard way to express this intent. mypy will say that's an error (if
there are type annotations), but we get the other problem of *allowing*
monkey patching.
StackOverflow suggests
<http://stackoverflow.com/questions/35988/c-like-structures-in-python> either
NamedTuple (which is not what I want in this case, since it is immutable
and indexable), another answer without __init__ (which requires useful
default values, and no sign of the "non-mankey-patch" intention). another
answer with arbitrary **kwargs assignment (very flexible, but zero
reasoning). Yet another suggests `dict` which is useful but has nothing to
do with reasoning.
There are many solutions in pypi, most only address the boilerplate issue;
but the "attrs" package seems nice; it might fit in the alternative
decorator-oriented framework I mentioned.
There isn't even a standard name for that feature. "Struct" means something
else.

* Immutable: The importance to reasoning is obvious. There are many
alternative suggestions; again, most upvoted is NamedTuple, which is great
but also indexable. This means one kind of type error that will not be
caught. (actually two, since the __init__ method takes positional arguments
that can be mixed). Additionally, inheriting classes can be monkey-patched,
and there's no standard way to disallow inheritance. The above problems can
be solved in an ad-hoc manner, but it won't give tool support. Other
solutions involve messing up with __setattr__ etc. It Again, "attrs" seems
has a solution in the form of "freeze".

* Sealed: Useful for typechecker (prevent erroneus cast) and optimizers
that need to know the actual implementation of method dispatch (I believe
it can be used in CPython too, but that's a different topic).
The only solution I've found is this answer
http://stackoverflow.com/questions/16564198/pythons-equivalent-of-nets-sealed-class
which gives a hand-crafeted metaclass. But if it's not conventional, it
won't be used by public tools.

* Array: If I want a tuple with mutable cells, I currently simply don't
have any option. There's list, which is extendable. There's array.array and
numpy's matrices, which are intended for numeric processing. There might be
some package for that, but I can't find it - and of course none that allow
type annotations. Besides, there's no reason to assume it will work
together with 

Re: [Python-ideas] typing.modifiers

[אלעזר]

Thanks for the reply

בתאריך יום ו׳, 16 בספט' 2016, 13:16, מאת Steven D'Aprano ‏<
st...@pearwood.info>:

> On Fri, Sep 16, 2016 at 12:10:22AM +0000, אלעזר wrote:
>
> [...]
> > Benefits of putting such a collection in stdlib (instead of as an
> external
> > package) include:
>
> Slow down! Before getting all excited about adding these typing hints(?)
> into typing.modifiers, you first have to convince people that they are
> useful and deserve a place in the std library.
>
I was thinking this is the place to do this?

What problem are these hints/classes supposed to solve? What solutions
> already exist? Why aren't those solutions good enough?
>
I have addressed that only very briefly; I will try to elaborate later (I'm
writing from the phone. Sorry)

>
> > 1. This information can be used by typecheckers, and also by users, to
> > reason about programs. If isinstance(x, ImmutableArray), then x is an
> > instantiation of ImmutableArray.
>
> That's how type-checkers work. The class doesn't need to be in the std
> lib for a type-checker to reason about it.
>
No, it's not how they work, since it's not true. I meant the actual type,
not a subtype.

>
>
> > 2. A conventional syntax and a single answer for "How do I make my class
> > immutable", "How do I make my class unsubclassable"
>
> Do we need syntax for those?
>
>
> > 3. The syntax, especially for Struct as above, is pretty and clean. The
> > Array syntax is less so.
>
> I don't even understand what the Array syntax is supposed to mean.
>
That's already a bad sign... Each underscore gives type to its index. The
last index is the maximal.

>
> > 4. I think that the array implementation can use internal CPython details
> > to be implemented efficiently.
>
> What happens to Jython, PyPy, IronPython etc?
>
Similarly or even more so.


> > I am not sure that typing.modifiers is the right place, since these are
> not
> > exactly type hints; they generate methods, and are intended to be
> enforced
> > at runtime.
>
> Then I think your answer is: no, typing.modifiers is NOT the right
> place.
>
Speculating it will make its way, where should it land then?

>
> > I think that even if this idea is not accepted, the general theme is
> > something that might be useful to keep in mind, stdlib might accumulate
> > such modifiers, and it will be nice to keep things uniform.
>
> The stdlib might accumulate many things. Why should it accumulate these?
>
In this part I wasn't talking about what should happen, but rather what
might happen gradually, in which case it'll be nice to fit in a uniform
structure.

>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] typing.modifiers

[אלעזר]

With the new variable annotation syntax, it is possible to implement a
useful "modifiers" library, all used as superclasses.

Possible modifiers:
  * Named / Struct: annotation-defined fields. No monkey patching.
  * Immutable / Const
  * Sealed / Final: unsubclassable class
  * Array: a mutable tuple. (No "+" or "*" operators)

There of course may be others. Subclasses of the modifiers get default
methods and behavior.
The modifiers can used in combinations. For example:

class Command(Immutable, Named):
cmd : str = "NOP"
value : int = None

load = Command(cmd="load", value=1000)  # no positional arguments

The syntax for Array is less pretty:

class Command(Array):
_0 : str = "NOP"
_1 : int = None

load = Command("load", 1000)  # no positional arguments

Many other combinations are useful too. "Sealed" is almost orthogonal to
the others,  Obviously, some combinations already exists:
  * (Immutable, Array) is similar to Tuple
  * NamedTuple is another name for (Immutable, Named, Array).
  * Enum should be on this list too

Some less promising modifiers:
  * Namespace / Static: uninstantiable class. A module.
  * Volatile, for externally facing classes. Hints the static checkers /
jitters that they should not assume they know the values.
 * Abstract

Alternatives and cons :
I suggest base classes instead of decorators since NamedTuple and tuple go
this way, and since it is static information. I am not sure which is better
though.
Metaclass parameters can be used (and actually used in my implementation,
and in NamedTuple's), but the syntax is uglier.
"array" is only for numeric values in Python, so this name is
problematic.So is struct.

I have a rough implementation for most of the above; much of it is not hard
in general, though some details are hard to get right.
---

Benefits of putting such a collection in stdlib (instead of as an external
package) include:
1. This information can be used by typecheckers, and also by users, to
reason about programs. If isinstance(x, ImmutableArray), then x is an
instantiation of ImmutableArray.
2. A conventional syntax and a single answer for "How do I make my class
immutable", "How do I make my class unsubclassable"
3. The syntax, especially for Struct as above, is pretty and clean. The
Array syntax is less so.
4. I think that the array implementation can use internal CPython details
to be implemented efficiently.

I am not sure that typing.modifiers is the right place, since these are not
exactly type hints; they generate methods, and are intended to be enforced
at runtime.

I think that even if this idea is not accepted, the general theme is
something that might be useful to keep in mind, stdlib might accumulate
such modifiers, and it will be nice to keep things uniform.

Opinions?

~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Generics Syntax

[אלעזר]

Yes the "class A[T]:" syntax requires on the ability to express variance as
an operator, but not the other way around.

It might be an argument in favor of switching to the + syntax: to make
possible future syntax change in class definition somewhat easier to
swallow.

~Elazar

On Thu, Sep 15, 2016 at 2:03 PM Nick Coghlan <ncogh...@gmail.com> wrote:

> On 15 September 2016 at 19:53, Ivan Levkivskyi <levkivs...@gmail.com>
> wrote:
> >
> >
> > On 15 September 2016 at 11:46, אלעזר <elaz...@gmail.com> wrote:
> >>
> >> And that thread is only about variance. What about the generic syntax?
> >
> >
> >  If you mean code like this:
> >
> > class Container[+T]:
> > @abstractmethod
> > def __contains__(self, x: T) -> bool: ...
> >
> > then there is little chance that this will be accepted because it
> requires
> > changes to Python syntax.
>
> If the proposed spelling is tweaked to be "class
> Container(Generic[+T]):", then it doesn't require a syntax change, as
> that's merely a matter of implementing unary plus on type vars:
>
> >>> +object()
> Traceback (most recent call last):
>  File "", line 1, in 
> TypeError: bad operand type for unary +: 'object'
> >>> class UnaryPlus:
> ... def __pos__(self):
> ... return self
> ...
> >>> +UnaryPlus()
> <__main__.UnaryPlus object at 0x7f5e0fe91c50>
>
> (I have no opinion on the value of providing a simpler spelling for
> covariance, I'm just noting that if you keep the "Generic[T]" aspect
> of the current spelling it wouldn't require any changes to Python's
> syntax and will work as far back as you care to support it)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Generics Syntax

[אלעזר]

And that thread is only about variance. What about the generic syntax?

‪On Thu, Sep 15, 2016 at 12:44 PM ‫אלעזר‬‎ <elaz...@gmail.com> wrote:‬

> Does that mean that if I did, it would be reconsidered?
>
> On Thu, Sep 15, 2016 at 12:43 PM Ivan Levkivskyi <levkivs...@gmail.com>
> wrote:
>
>> On 15 September 2016 at 11:21, אלעזר <elaz...@gmail.com> wrote:
>>
>>> This suggestion is so obvious that it's likely has been discussed, but I
>>> can't find any reference (It's not what PEP-3124 talks about).
>>>
>>>
>> You might want to read this tread
>> https://github.com/python/typing/issues/211 and another tread mentioned
>> there at the start.
>> In short, this idea has been discussed, but nobody had seen it as good
>> enough to sacrifice his time for implementing the changes.
>>
>> --
>> Ivan
>>
>>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Generics Syntax

[אלעזר]

Does that mean that if I did, it would be reconsidered?

On Thu, Sep 15, 2016 at 12:43 PM Ivan Levkivskyi <levkivs...@gmail.com>
wrote:

> On 15 September 2016 at 11:21, אלעזר <elaz...@gmail.com> wrote:
>
>> This suggestion is so obvious that it's likely has been discussed, but I
>> can't find any reference (It's not what PEP-3124 talks about).
>>
>>
> You might want to read this tread
> https://github.com/python/typing/issues/211 and another tread mentioned
> there at the start.
> In short, this idea has been discussed, but nobody had seen it as good
> enough to sacrifice his time for implementing the changes.
>
> --
> Ivan
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Generics Syntax

[אלעזר]

This suggestion is so obvious that it's likely has been discussed, but I
can't find any reference (It's not what PEP-3124 talks about).

Generic class syntax, now:

_T_co = TypeVar('_T', covariant=True)

class Container(Generic[_T_co]):
@abstractmethod
def __contains__(self, x: _T_co) -> bool: ...

(yes it's object in reality)
Generic class syntax, suggested:

class Container[+T]:
@abstractmethod
def __contains__(self, x: T) -> bool: ...

The + signifies covariant type, as in Scala. The need for underscore prefix
and explicit name for covariant types is gone, since it's class-scoped.
The + is a bit cryptic, but so are term "covariant" and Generic[_T_co]. The
syntax by large is exactly what users coming from statically-typed
languages will expect.

Giving a bound can be done using "T <: AnyStr", "T: AnyStr" or any other
syntax.

Again, I assume it was discussed before, but my Google skills are failing
me; a pointer will be helpful.

~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] if-statement in for-loop

[אלעזר]

This has come up before. It will be a special case of making "if" without
"else" result in a special "empty" type that is not part of the iteration.
As in `[1, (2 if False) ] == [1]`.

בתאריך יום א׳, 11 בספט' 2016, 13:29, מאת Bernardo Sulzbach ‏<
mafagafogiga...@gmail.com>:

> On 09/11/2016 06:36 AM, Dominik Gresch wrote:
> > So I asked myself if a syntax as follows would be possible:
> >
> > for i in range(10) if i != 5:
> > body
> >
> > Personally, I find this extremely intuitive since this kind of
> > if-statement is already present in list comprehensions.
> >
> > What is your opinion on this? Sorry if this has been discussed before --
> > I didn't find anything in the archives.
> >
>
> I find it interesting.
>
> I thing that this will likely take up too many columns in more
> convoluted loops such as
>
>  for element in collection if is_pretty_enough(element) and ...:
>  ...
>
> However, this "problem" is already faced by list comprehensions, so it
> is not a strong argument against your idea.
>
> I am still unsure about whether or not the pattern you describe is
> frequent enough to justify special syntax. Not to mention that the
> current way to do it is already **very** readable. Just notice how
>
>  for e in l:
>  if e != 2:
>  ...
>
> and
>
>  for e in l if e != 2:
>  ...
>
> read essentially the same and take about the same number of keystrokes.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[אלעזר]

(Just to be clear, I wasn't trying to suggest this as more than an ad-hoc
solution for a throwaway script. But to me, "sorted by random key" is
almost as obvious as "shuffled", perhaps more so for non english speakers
with little background in CS terms; the words "sorted" and "random" jumps
to the eye, and sometimes you don't need more than that)
~Elazar

בתאריך יום ג׳, 6 בספט' 2016, 21:48, מאת Bernardo Sulzbach ‏<
mafagafogiga...@gmail.com>:

> On 09/06/2016 03:37 PM, Sven R. Kunze wrote:
> >
> > Besides being a silly argument, it's an interesting solution.
> >
> > Does it really work? I remember Microsoft utilizing a similar approach
> > for their browser selection tool which led to a skewed probability
> > distribution. Maybe, I wrong here though.
> >
>
> Yes. The key is evaluated only once, so each element gets a
> pseudo-random number. Sorting this list leads to a shuffle.
>
> However, a super-linear shuffle, whilst Fisher-Yates is a linear
> solution and also requires less additional memory.
>
> Lastly, although it is obvious, from a software engineering standpoint,
> this is a mere weird hack.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[אלעזר]

Naive shuffled() can be emulated using a single expression:

   sorted(lst, key=lambda _: random())

So there's even less incentive for standardization.

~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] discontinue iterable strings

[אלעזר]

On Sun, Aug 21, 2016 at 12:28 AM Alexander Heger  wrote:

> Did I leave anything out?
>> How would you weigh the benefits against the problems?
>> How would you manage the upgrade path for code that's been broken?
>>
>
> FIrst one needs to add the extension string attributes like
> split()/split(''), chars(), and substring[] (Python 3.7).
>
> When indexing becomes disallowed (Python 3.10 / 4.0) attempts to iterate
> (or slice) will raise TypeError.  The fixes overall will be a lot easier
> and obvious than introduction of unicode as default string type in Python
> 3.0.  It could already be used/test starting with Python 3.7 using 'from
> future import __monolythic_strings__`.
>
>  Is there any equivalent __future__ import with such deep semantic
implications? Most imports I can think of are mainly syntactic.
And what would it do? change the type of string literals? change the
behavior of str methods locally in this module? globally? How will this
play with 3rd party libraries?
Sounds like it will break stuff in a way that cannot be locally fixed.

~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/