Re: [Python-Dev] What's the status of PEP 515?

[Ned Deily]

At the dev sprint today, we discussed PEP 515; several people are keen to see 
it get into 3.6. If someone doesn't get to it before tomorrow, one of the 
sprinters will try to do a final review and get it pushed.

  --
Ned Deily
n...@python.org -- []


> On Sep 6, 2016, at 21:21, Eric V. Smith  wrote:
> 
> The implementation of '_' in numeric literals is here:
> http://bugs.python.org/issue26331
> 
> And to add '_' in int.__format__ is here:
> http://bugs.python.org/issue27080
> 
> But I don't want to add support in int.__format__ unless numeric literal 
> support is added.
> 
> So, Georg and Serhiy: is issue 26331 going to get committed? If so, I'll 
> commit 27080 (or you can). I just don't want the second part of PEP 515 to 
> not make the deadline if the first part makes it in at the last minute.
> 
> Thanks!
> Eric.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/nad%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] What's the status of PEP 515?

[Eric V. Smith]

The implementation of '_' in numeric literals is here:
http://bugs.python.org/issue26331

And to add '_' in int.__format__ is here:
http://bugs.python.org/issue27080

But I don't want to add support in int.__format__ unless numeric literal 
support is added.


So, Georg and Serhiy: is issue 26331 going to get committed? If so, I'll 
commit 27080 (or you can). I just don't want the second part of PEP 515 
to not make the deadline if the first part makes it in at the last minute.


Thanks!
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Guido van Rossum]

On Tue, Sep 6, 2016 at 8:25 AM, Mark Shannon  wrote:
> The "smartness" of checkers is not the problem (for this example, at least)
> the problem is that checkers must conform to the rules laid down in PEP 484
> and (in whatever form it finally takes) PEP 526.
> It sounds like mypy doesn't conform to PEP 526, as it ignoring the declared
> type of x and using the inferred type.
> In fact it looks as if it is doing exactly what I proposed, which is that
> the annotation describes the type of the expression, not the variable.

IMO neither PEP requires type checkers to behave this way. Maybe you
read it between the lines when you reviewed PEP 484 and neither of us
realized that we were interpreting the text differently? The words you
have quoted previously mean different things to me than you seem to
imply.

>> I guess this is a surprise if you think of type systems like Java's
>> where the compiler forgets what it has learned, at least from the
>> language spec's POV. But a Python type checker is more like a linter,
>> and false positives (complaints about valid code) are much more
>> problematic than false negatives (passing invalid code).

> The language of PEP 526 is strongly suggestive of a type system like Java.

That suggestion is really in your mind. The PEP also quite clearly
states that it does not specify what a type checker should do with the
"declarations".

> The extensive use of the term 'variable' rather than 'expression' and
> 'assignment' rather suggests that all definitions and uses of a single
> variable have the same type.

Maybe you believe that Python's use of the word 'variable', combined
with using `=` for assignment, also implies that Python's "variables"
should behave like Java's "variables"?

> The problem with using the term "variable" is that it is *not* vague.
> Variables in python have well defined scopes and lifetimes.

So? When a type checker can prove that in the expression `f(x)`, the
type of the *expression* `x` will be compatible with the argument type
expected by f, isn't that good enough? Why would the type given for
the *variable* `x` have to be the only input to the type check for
that expression?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 525, fourth update

[Guido van Rossum]

Thanks Yury!

I am hereby accepting PEP 525 provisionally. The acceptance is so that
you can go ahead and merge this into 3.6 before the feature freeze
this weekend. The provisional status is because this is a big project
and it's likely that we'll need to tweak some small aspect of the API
once the code is in, even after 3.6.0 is out. (Similar to the way PEP
492, async/await, was accepted provisionally.) But I am cautiously
optimistic and I am grateful to Yury for the care and effort he has
put into it.

--Guido

On Tue, Sep 6, 2016 at 5:10 PM, Yury Selivanov  wrote:
> Hi,
>
> I've updated PEP 525 with a new section about asyncio changes.
>
> Essentially, asyncio event loop will get a new "shutdown_asyncgens" method
> that allows to close the loop and all associated AGs with it reliably.
>
> Only the updated section is pasted below:
>
>
> asyncio
> ---
>
> The asyncio event loop will use ``sys.set_asyncgen_hooks()`` API to
> maintain a weak set of all scheduled asynchronous generators, and to
> schedule their ``aclose()`` coroutine methods when it is time for
> generators to be GCed.
>
> To make sure that asyncio programs can finalize all scheduled
> asynchronous generators reliably, we propose to add a new event loop
> method ``loop.shutdown_asyncgens(*, timeout=30)``.  The method will
> schedule all currently open asynchronous generators to close with an
> ``aclose()`` call.
>
> After calling the ``loop.shutdown_asyncgens()`` method, the event loop
> will issue a warning whenever a new asynchronous generator is iterated
> for the first time.  The idea is that after requesting all asynchronous
> generators to be shutdown, the program should not execute code that
> iterates over new asynchronous generators.
>
> An example of how ``shutdown_asyncgens`` should be used::
>
> try:
> loop.run_forever()
> # or loop.run_until_complete(...)
> finally:
> loop.shutdown_asyncgens()
> loop.close()
>
> -
> Yury
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The Amazing Unreferenced Weakref

[Gregory P. Smith]

This code appears to have been added to fix
https://bugs.python.org/issue3100 - A crash involving a weakref subclass.

-gps

On Tue, Sep 6, 2016 at 3:51 PM Larry Hastings  wrote:

>
> This is all about current (3.6) trunk.
>
> In Objects/weakrefobject.c, we have the function
> PyObject_ClearWeakRefs().  This is called when a generic object that
> supports weakrefs is destroyed; this is the code that calls the callbacks.
> Here's a little paragraph of code from the center:
>
> for (i = 0; i < count; ++i) {
> PyWeakReference *next = current->wr_next;
>
> if (((PyObject *)current)->ob_refcnt > 0)
> {
> Py_INCREF(current);
> PyTuple_SET_ITEM(tuple, i * 2, (PyObject *) current);
> PyTuple_SET_ITEM(tuple, i * 2 + 1, current->wr_callback);
> }
> else {
> Py_DECREF(current->wr_callback);
> }
> current->wr_callback = NULL;
> clear_weakref(current);
> current = next;
> }
>
> "current" is the doubly-linked list of PyWeakReference objects stored
> inside the object that's getting destroyed.
>
> My question: under what circumstances would ob_refcnt ever be 0?  The
> tp_dealloc handler for PyWeakReference * objects removes it from this list
> and frees the memory.  How could the reference count reach 0 without
> tp_dealloc being called and it being removed from the list?
>
> Scratching my head like crazy,
>
>
> */arry*
>
> p.s. If you're thinking "why does he care?", understanding this would
> maybe help with the Gilectomy.  So yes there's a point to this question.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 525, fourth update

[Yury Selivanov]

Hi,

I've updated PEP 525 with a new section about asyncio changes.

Essentially, asyncio event loop will get a new "shutdown_asyncgens" 
method that allows to close the loop and all associated AGs with it 
reliably.


Only the updated section is pasted below:


asyncio
---

The asyncio event loop will use ``sys.set_asyncgen_hooks()`` API to
maintain a weak set of all scheduled asynchronous generators, and to
schedule their ``aclose()`` coroutine methods when it is time for
generators to be GCed.

To make sure that asyncio programs can finalize all scheduled
asynchronous generators reliably, we propose to add a new event loop
method ``loop.shutdown_asyncgens(*, timeout=30)``.  The method will
schedule all currently open asynchronous generators to close with an
``aclose()`` call.

After calling the ``loop.shutdown_asyncgens()`` method, the event loop
will issue a warning whenever a new asynchronous generator is iterated
for the first time.  The idea is that after requesting all asynchronous
generators to be shutdown, the program should not execute code that
iterates over new asynchronous generators.

An example of how ``shutdown_asyncgens`` should be used::

try:
loop.run_forever()
# or loop.run_until_complete(...)
finally:
loop.shutdown_asyncgens()
loop.close()

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] A Pseudo-Post-Mortem (not dead yet) on my Multi-Core Python Project.

[Eric Snow]

I'm not anticipating much discussion on this, but wanted to present a
summary of my notes from the project I proposed last year and have
since tabled.

http://ericsnowcurrently.blogspot.com/2016/09/solving-mutli-core-python.html

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 447: Add __getdescriptor__ to metaclasses

[Guido van Rossum]

Hi Ronald,

The feature freeze for 3.6 is closing in a few days; 3.6b1 will go out
this weekend. Did you overcome the issue, or does your PEP need to be
postponed until 3.7?

--Guido

On Sun, Jul 24, 2016 at 9:58 PM, Ronald Oussoren  wrote:
>
> On 24 Jul 2016, at 13:06, Ronald Oussoren  wrote:
>
> …
>
> But on the other hand, that’s why wanted to use PyObjC to validate
> the PEP in the first place.
>
>
> I’ve hit a fairly significant issue with this, PyObjC’s super contains more
> magic than just this magic that would be fixed by PEP 447. I don’t think
> I’ll be able to finish work on PEP 447 this week because of that, and in the
> worst case will have to retire the PEP.
>
> The problem is as follows: to be able to map all of Cocoa’s methods to
> Python PyObjC creates two proxy classes for every Cocoa class: the regular
> class and its metaclass. The latter is used to store class methods. This is
> needed because Objective-C classes can have instance and class methods with
> the same name, as an example:
>
> @interface NSObject
> -(NSString*)description;
> +(NSString*)description
> @end
>
> The first declaration for “description” is an instance method, the second is
> a class method.  The Python metaclass is mostly a hidden detail, users don’t
> explicitly interact with these classes and use the normal Python convention
> for defining class methods.
>
> This works fine, problems starts when you want to subclass in Python and
> override the class method:
>
> class MyClass (NSObject):
>@classmethod
>def description(cls):
>   return “hello there from %r” % (super(MyClass, cls).description())
>
> If you’re used to normal Python code there’s nothing wrong here, but getting
> this to work required some magic in objc.super to ensure that its
> __getattribute__ looks in the metaclass in this case and not the regular
> class.  The current PEP447-ised version of PyObjC has a number of test
> failures because builtin.super obviously doesn’t contain this hack (and
> shouldn’t).
>
> I think I can fix this for modern code that uses an argumentless call to
> super by replacing the cell containing the __class__ reference when moving
> the method from the regular class to the instance class. That would
> obviously not work for the code I showed earlier, but that at least won’t
> fail silently and the error message is specific enough that I can include it
> in PyObjC’s documentation.
>
> Ronald
>
>
>
>
>
> Back to wrangling C code,
>
>   Ronald
>
>
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] The Amazing Unreferenced Weakref

[Larry Hastings]

This is all about current (3.6) trunk.

In Objects/weakrefobject.c, we have the function 
PyObject_ClearWeakRefs().  This is called when a generic object that 
supports weakrefs is destroyed; this is the code that calls the 
callbacks.  Here's a little paragraph of code from the center:


   for (i = 0; i < count; ++i) {
PyWeakReference *next = current->wr_next;

if (((PyObject *)current)->ob_refcnt > 0)
{
Py_INCREF(current);
PyTuple_SET_ITEM(tuple, i * 2, (PyObject *) current);
PyTuple_SET_ITEM(tuple, i * 2 + 1, current->wr_callback);
}
else {
Py_DECREF(current->wr_callback);
}
current->wr_callback = NULL;
clear_weakref(current);
current = next;
   }

"current" is the doubly-linked list of PyWeakReference objects stored 
inside the object that's getting destroyed.


My question: under what circumstances would ob_refcnt ever be 0? The 
tp_dealloc handler for PyWeakReference * objects removes it from this 
list and frees the memory.  How could the reference count reach 0 
without tp_dealloc being called and it being removed from the list?


Scratching my head like crazy,


//arry/

p.s. If you're thinking "why does he care?", understanding this would 
maybe help with the Gilectomy.  So yes there's a point to this question.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Chris Angelico]

On Wed, Sep 7, 2016 at 4:11 AM, Stephen J. Turnbull
 wrote:
> Finally, the notion of annotating expressions is incoherent:
>
> # Annotating (sub)expressions: the more the merrier!
> (x) : bool = (((y): int + (z): float) / (w): complex): quarternion
> # Ooh, an expression with no past and no future.  Annotate it!
> (y + z) / w: quarternion

Can't do that - parsing would become ambiguous.

x = {1:int, 1.5:float, 2+3j:complex}
print(type(x))

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Stephen J. Turnbull]

Mark Shannon writes:

 > The problem with using the term "variable" is that it is *not* vague. 
 > Variables in python have well defined scopes and lifetimes.

Sure, but *hints* are not well-defined by Python (except the syntax,
once PEP 526 is implemented).  A *hint* is something that the
typechecker takes note of, and then does whatever it pleases with it.

So can we do the practical thing here and agree that even though the
type hint on a variable is constant, what the typechecker does with
that type hint in different contexts might change?


The rest is tl;dr (why I want type hints on variables, and why the
term "annotating expressions" leaves me cold).

I don't see how you can interpret

z: complex = 1.0

without a notion of annotating the variable.  The RHS is clearly of
float type, and Python will assign a float to z.  What's going on
here?  Maybe this:

from math import exp

z: complex = 1.0
print(exp(z))

==> "MyPy to Steve!  MyPy to Steve!  You are confused!"

Maybe "math" was a typo for "cmath".  Maybe "complex" was a premature
generalization.  Maybe you wouldn't want to hear about it ... but I
would.  I think.  Anyway, to find out if I *really* want that or not,
I need a notion of hinting the variable.

But: "although practicality beats purity".  Like everybody else, I
want a typechecker that minds its manners and says nothing about

from math import exp

z: complex = 1.0
try:
print(exp(z))
except TypeError:
print("Oh well, complex is better than complicated.")

Finally, the notion of annotating expressions is incoherent:

# Annotating (sub)expressions: the more the merrier!
(x) : bool = (((y): int + (z): float) / (w): complex): quarternion
# Ooh, an expression with no past and no future.  Annotate it!
(y + z) / w: quarternion

Noone has any intention of annotating expressions AFAICS -- people who
talk about that really mean annotating the *value* of the expression
on the RHS, but since it will *always* be on the RHS of an assignment,
it's equivalent to annotating the value of the *target* on the LHS.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Ivan Levkivskyi]

On 6 September 2016 at 18:35, Nick Coghlan  wrote:

> On 7 September 2016 at 01:33, Ivan Levkivskyi 
> wrote:
> > On 6 September 2016 at 17:25, Mark Shannon  wrote:
> >>
> >> The issue is not whether the checker can tell that the type of the
> >> *expression* is int, but whether it is forced to use the type of the
> >> *variable*. The current wording of PEP 526 strongly implies the latter.
> >
> > Mark,
> > Could you please point to exact locations in the PEP text and propose an
> > alternative wording, so that we will have a more concrete discussion.
>
> Rather than trying to work that out on the list, it may make the most
> sense for Mark to put together a PR that rewords the parts of the PEP
> that he sees as constraining typecheckers to restrict *usage* of a
> variable based on its annotation, rather than just restricting future
> bindings to it.
>

Thanks Nick, this is a good idea.
Mark, I will be glad to discuss your PR to the master python/peps repo.

--
Iavn
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting on python directories

[Steven D'Aprano]

Hello Ramu,

This is the wrong place to ask for help with your question, this is for 
development of the Python interpreter.

I suggest you subscribe to the Python-List mailing list.

For help with publishing packages, see 

https://wiki.python.org/moin/CheeseShopTutorial



On Tue, Sep 06, 2016 at 10:01:36PM +0530, RAMU V wrote:
> Sir
>   I learn the basics of python from online and I am pretty much confident 
> with my basics and I want to show them to the outside world by publishing 
> them as packages in python.  There were set of instructions for that process 
> but I could not figure out the exact process so please help me with this 
> because I learnt  python out of my interest but not based on my syllabus or 
> for job. Its like a refreshing enjoyment for my brain.
> Sent from Mail for Windows 10



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting on python directories

[Ryan Gonzalez]

Wrong mailing list. This is for the discussion of development *of* Python,
not *in* Python. You probably want:

https://mail.python.org/mailman/listinfo/python-list

Regardless, this page should answer your questions:

https://packaging.python.org/distributing/


On Tue, Sep 6, 2016 at 11:31 AM, RAMU V  wrote:

> Sir
>
> I learn the basics of python from online and I am pretty
> much confident with my basics and I want to show them to the outside world
> by publishing them as packages in python.  There were set of instructions
> for that process but I could not figure out the exact process so please
> help me with this because I learnt  python out of my interest but not based
> on my syllabus or for job. Its like a refreshing enjoyment for my brain.
>
> Sent from Mail  for
> Windows 10
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> rymg19%40gmail.com
>
>


-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Nick Coghlan]

On 7 September 2016 at 01:33, Ivan Levkivskyi  wrote:
> On 6 September 2016 at 17:25, Mark Shannon  wrote:
>>
>> The issue is not whether the checker can tell that the type of the
>> *expression* is int, but whether it is forced to use the type of the
>> *variable*. The current wording of PEP 526 strongly implies the latter.
>
> Mark,
> Could you please point to exact locations in the PEP text and propose an
> alternative wording, so that we will have a more concrete discussion.

Rather than trying to work that out on the list, it may make the most
sense for Mark to put together a PR that rewords the parts of the PEP
that he sees as constraining typecheckers to restrict *usage* of a
variable based on its annotation, rather than just restricting future
bindings to it.

It seems to me everyone's actually in pretty good agreement on how we
want variable annotations to work (constraining future assignments to
abide by the declaration, without constraining use when inference
indicates a more specific type), but the PEP may be using some
particular terminology more loosely than is strictly correct in the
context of type theory.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Requesting on python directories

[RAMU V]

Sir
I learn the basics of python from online and I am pretty much confident 
with my basics and I want to show them to the outside world by publishing them 
as packages in python.  There were set of instructions for that process but I 
could not figure out the exact process so please help me with this because I 
learnt  python out of my interest but not based on my syllabus or for job. Its 
like a refreshing enjoyment for my brain.
Sent from Mail for Windows 10

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Guido van Rossum]

On Tue, Sep 6, 2016 at 9:00 AM, Nick Coghlan  wrote:
> On 6 September 2016 at 14:04, Guido van Rossum  wrote:
>> I'm sorry, but we're not going to invent new syntax this late in the
>> game. The syntax proposed by the PEP has been on my mind ever since
>> PEP 484 with very minor variations; I first proposed it seriously on
>> python-ideas over a month ago, we've been debating the details since
>> then, and it's got a solid implementation based on those debates by
>> Ivan Levkivskyi. In contrast, it looks like you just made the "assert
>> x: T" syntax up last night in response to the worries expressed by
>> Mark Shannon, and "assert" sounds a lot like a run-time constraint to
>> me.
>
> That's a fair description, but the notation also helped me a lot in
> articulating the concepts I was concerned about without having to put
> dummy annotated functions everywhere :)

Thanks Nick! It seems your writings has helped some others (e.g.
Ethan) understand PEP 526.

>> Instead, I encourage you to participate in the writing of a separate
>> PEP explaining how type checkers are expected to work (since PEP 526
>> doesn't specify that). Ivan is also interested in such a PEP and we
>> hope Mark will also lend us his expertise.
>
> Aye, I'd be happy to help with that - I think everything proposed can
> be described in terms of existing PEP 484 primitives and the
> descriptor protocol, so the requirements on typecheckers would just be
> for them to be self-consistent, rather than defining fundamentally new
> behaviours.

Beware that there are by now some major type checkers that already
claim conformance to PEP 484 in various ways: mypy, pytype, PyCharm,
and probably Semmle.com where Mark works has one too. Each one has
some specialty and each one is a work in progress, but a PEP shouldn't
start out by declaring the approach used by any existing checker
unlawful.

As an example, mypy doesn't yet support Optional by default: it
recognizes the syntax but it doesn't distinguish between e.g. int and
Optional[int]. (It will do the right thing when you pass the
`--strict-optional` flag, but there are still some issues with that
before we can make it the default behavior.)

As another example: mypy understands isinstance() checks so that e.g.
the following works:

def foo(x: Union[int, str]) -> str:
if isinstance(x, str):
return x
return str(x)

I don't think you can find anything in PEP 484 that says this should
work; but without it mypy would be much less useful. (The example here
is silly, but such code appears in real life frequently.)

One final thought: this is not the first time that Python has used
syntax that looks like another language but gives it a different
meaning. In fact, apart from `if`, almost everything in Python works
differently than it works in C++ or Java. So I don't worry much about
that.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 6 September 2016 at 02:51, Ian Foote  wrote:
> On 05/09/16 14:46, Nick Coghlan wrote:
>> That's not what the PEP proposes for uninitialised variables though:
>> it proposes processing them *before* a series of assignment
>> statements, which *only makes sense* if you plan to use them to
>> constrain those assignments in some way.
>>
>> If you wanted to write something like that under a type assertion
>> spelling, then you could enlist the aid of the "all" builtin:
>>
>> assert all(x) : List[T] # All local assignments to "x" must abide
>> by this constraint
>> if case1:
>> x = ...
>> elif case2:
>> x = ...
>> else:
>> x = ...
>>
>
> Would the `assert all(x)` be executed at runtime as well or would this
> be syntax only for type checkers? I think this particular spelling at
> least is potentially confusing.

Only for typecheckers, same as the plans for function level bare
annotations. Otherwise it wouldn't work, since you'd be calling
"all()" on a non-iterable :)

Guido doesn't like the syntax though, so the only place it would ever
appear is explanatory notes describing the purpose of the new syntax,
and hence can be replaced by something like:

# After all future assignments to x, check that x conforms to T

Cheers,
Nick.

P.S. Or, if you're particularly fond of mathematical notation, and we
take type categories as sets:

# ∀x: x ∈ T

That would be a singularly unhelpful explanatory comment for the vast
majority of folks, though :)

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 6 September 2016 at 14:04, Guido van Rossum  wrote:
> I'm sorry, but we're not going to invent new syntax this late in the
> game. The syntax proposed by the PEP has been on my mind ever since
> PEP 484 with very minor variations; I first proposed it seriously on
> python-ideas over a month ago, we've been debating the details since
> then, and it's got a solid implementation based on those debates by
> Ivan Levkivskyi. In contrast, it looks like you just made the "assert
> x: T" syntax up last night in response to the worries expressed by
> Mark Shannon, and "assert" sounds a lot like a run-time constraint to
> me.

That's a fair description, but the notation also helped me a lot in
articulating the concepts I was concerned about without having to put
dummy annotated functions everywhere :)

> Instead, I encourage you to participate in the writing of a separate
> PEP explaining how type checkers are expected to work (since PEP 526
> doesn't specify that). Ivan is also interested in such a PEP and we
> hope Mark will also lend us his expertise.

Aye, I'd be happy to help with that - I think everything proposed can
be described in terms of existing PEP 484 primitives and the
descriptor protocol, so the requirements on typecheckers would just be
for them to be self-consistent, rather than defining fundamentally new
behaviours.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ian Foote]

On 05/09/16 14:46, Nick Coghlan wrote:
> That's not what the PEP proposes for uninitialised variables though:
> it proposes processing them *before* a series of assignment
> statements, which *only makes sense* if you plan to use them to
> constrain those assignments in some way.
> 
> If you wanted to write something like that under a type assertion
> spelling, then you could enlist the aid of the "all" builtin:
> 
> assert all(x) : List[T] # All local assignments to "x" must abide
> by this constraint
> if case1:
> x = ...
> elif case2:
> x = ...
> else:
> x = ...
> 

Would the `assert all(x)` be executed at runtime as well or would this
be syntax only for type checkers? I think this particular spelling at
least is potentially confusing.

Regards,
Ian F



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Ivan Levkivskyi]

On 6 September 2016 at 17:25, Mark Shannon  wrote:

> The issue is not whether the checker can tell that the type of the
> *expression* is int, but whether it is forced to use the type of the
> *variable*. The current wording of PEP 526 strongly implies the latter.
>

Mark,
Could you please point to exact locations in the PEP text and propose an
alternative wording, so that we will have a more concrete discussion.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Mark Shannon]

On 05/09/16 18:40, Guido van Rossum wrote:

On Mon, Sep 5, 2016 at 8:26 AM, Mark Shannon  wrote:

PEP 526 states that "This PEP aims at adding syntax to Python for annotating
the types of variables" and Guido seems quite insistent that the
declarations are for the types of variables.

However, I get the impression that most (all) of the authors and proponents
of PEP 526 are quite keen to emphasise that the PEP in no way limits type
checkers from doing what they want.

This is rather contradictory. The behaviour of a typechecker is defined by
the typesystem that it implements. Whether a type annotation determines the
type of a variable or an expression alters changes what typesystems are
feasible. So, stating that annotations define the type of variables *does*
limit what a typechecker can or cannot do.

Unless of course, others may have a different idea of what the "type of a
variable" means.
To me, it means it means that for all assignments `var = expr`
the type of `expr` must be a subtype of the variable,
and for all uses of var, the type of the use is the same as the type of the
variable.

In this example:

 def bar()->Optional[int]: ...

 def foo()->int:
 x:Optional[int] = bar()
 if x is None:
 return -1
 return x

According to PEP 526 the annotation `x:Optional[int]`
means that the *variable* `x` has the type `Optional[int]`.
So what is the type of `x` in `return x`?
If it is `Optional[int]`, then a type checker is obliged to reject this
code. If it is `int` then what does "type of a variable" actually mean,
and why aren't the other uses of `x` int as well?


Oh, there is definitely a problem here if you interpret it that way.
Of course I assume that other type checkers are at least as smart as
mypy. :-) In mypy, the analysis of this example narrows the type x can
have once `x is None` is determined to be false, so that the example
passes.


The "smartness" of checkers is not the problem (for this example, at 
least) the problem is that checkers must conform to the rules laid down 
in PEP 484 and (in whatever form it finally takes) PEP 526.
It sounds like mypy doesn't conform to PEP 526, as it ignoring the 
declared type of x and using the inferred type.
In fact it looks as if it is doing exactly what I proposed, which is 
that the annotation describes the type of the expression, not the variable.




I guess this is a surprise if you think of type systems like Java's
where the compiler forgets what it has learned, at least from the
language spec's POV. But a Python type checker is more like a linter,
and false positives (complaints about valid code) are much more
problematic than false negatives (passing invalid code).


The language of PEP 526 is strongly suggestive of a type system like 
Java. The extensive use of the term 'variable' rather than 'expression' 
and 'assignment' rather suggests that all definitions and uses of a 
single variable have the same type.




So a Python type checker that is to gain acceptance of users must be
much smarter than that, and neither PEP 484 not PEP 526 is meant to
require a type checker to complain about `return x` in the above
example.

I'm not sure how to change the language of the PEP though -- do you
have a suggestion? It all seems to depend on how the reader interprets
the meaning of very vague words like "variable" and "type".

The problem with using the term "variable" is that it is *not* vague. 
Variables in python have well defined scopes and lifetimes.



Cheers,
Mark.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Do PEP 526 type declarations define the types of variables or not?

[Mark Shannon]

On 05/09/16 23:16, Greg Ewing wrote:

Mark Shannon wrote:


Unless of course, others may have a different idea of what the "type
of a variable" means.
To me, it means it means that for all assignments `var = expr`
the type of `expr` must be a subtype of the variable,
and for all uses of var, the type of the use is the same as the type
of the variable.


I think it means that, at any given point in time, the
value of the variable is of the type of the variable or
some subtype thereof. That interpretation leaves the
type checker free to make more precise inferences if
it can. For example, in...


How does that differ from annotating the type of the expression?




def foo()->int:
x:Optional[int] = bar()
if x is None:
return -1
return x


...the type checker could notice that, on the branch
containing 'return x', the value of x must be of type
int, so the code is okay.



The issue is not whether the checker can tell that the type of the 
*expression* is int, but whether it is forced to use the type of the 
*variable*. The current wording of PEP 526 strongly implies the latter.


Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Koos Zevenhoven]

On Mon, Sep 5, 2016 at 6:06 AM, Nick Coghlan  wrote:
> On 5 September 2016 at 06:42, Koos Zevenhoven  wrote:
>> On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
>>>
>>> There are two self-consistent sets of names:
>>>
>>
>> Let me add a few. I wonder if this is really used so much that
>> bytes.chr is too long to type (and you can do bchr = bytes.chr if you
>> want to)
>>
>> bytes.chr (or bchr in builtins)
>
> The main problem with class method based spellings is that we need to
> either duplicate it on bytearray or else break the bytearray/bytes
> symmetry and propose "bytearray(bytes.chr(x))" as the replacement for
> current cryptic "bytearray([x])"

Warning: some API-design philosophy below:

1. It's not as bad to break symmetry regarding what functionality is
offered for related object types (here: str, bytes, bytearray) than it
is to break symmetry in how the symmetric functionality is provided.
IOW, a missing unnecessary functionality is less bad than exposing the
equivalent functionality under a different name. (This might be kind
of how Random832 was reasoning previously)

2. Symmetry is more important in object access functionality than it
is in instance creation. IOW, symmetry regarding 'constructors' (here:
bchr, bytes.chr, bytes.byte, ...) across different types is not as
crucial as symmetry in slicing. The reason is that the caller of a
constructor is likely to know which class it is instantiating. A
consumer of bytes/bytearray/str-like objects often does not know which
type is being dealt with.


I might be crying over spilled milk here, but that seems to be the
point of the whole PEP. That chars view thing might collect some of
the milk back back into a bottle:

mystr[whatever]  <-> mybytes.chars[whatever] <-> mybytearray.chars[whatever]
iter(mystr) <-> iter(mybytes.chars) <-> iter(mybytearray.chars)

Then introduce 'chars' on str and this becomes

mystring.chars[whatever]  <-> mybytes.chars[whatever] <->
mybytearray.chars[whatever]
iter(mystr.chars) <-> iter(mybytes.chars) <-> iter(mybytearray.chars)

If iter(mystr.chars) is recommended and iter(mystr) discouraged, then
after a decade or two, the world may look quite different regarding
how important it is for a str to be iterable.

This would solve multiple problems at once. Well I admit that "at
once" is not really an accurate description of the process :).

[...]
> You also run into a searchability problem as "chr" will get hits for
> both the chr builtin and bytes.chr, similar to the afalg problem that
> recently came up in another thread. While namespaces are a honking
> great idea, the fact that search is non-hierarchical means they still
> don't give API designers complete freedom to reuse names at will.

Oh, I can kind of see a point here, especially if the search hits
aren't related in any way. Why not just forget all symmetry if this is
an issue? But is it really a bad thing if by searching you find that
there's a chr for both str and bytes?

If I think, "I want to turn my int into a bytes 'character' kind of in
the way that chr turns my int into a str". What am I going to search
or google for? I can't speak for others, but I would probably search
for something that contains 'chr' and 'bytes'.

Based on this, I'm unable to see the search disadvantage of bytes.chr.

[...]
>> bytes.char(or bytes.chr or bchr in builtins)
>> bytes.chars, bytearray.chars (sequence views)
>
> The views are already available via memoryview.cast if folks really
> want them, but encouraging their use in general isn't a great idea, as
> it means more function developers now need to ask themselves "What if
> someone passes me a char view rather than a normal bytes object?".

Thanks, I think this is the first real argument I hear against the
char view. In fact, I don't think people should ask themselves that
question, and just not accept bytes views as input. Would it be enough
to discourage storing and passing bytes views?

Anyway, the only error that would pass silently would be that the
passed-in object gets indexed (e.g. obj[0]) and a bytes-char comes out
instead of an int. But it would be a strange thing to do by the caller
to pass a char view into the bytes-consumer. I could imagine someone
wanting to pass a bytes view into a str-consumer. But there are no
significant silently-passing errors there. If str also gets .chars,
then it becomes even easier to support this.

-- Koos

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

[Random832]

On Tue, Sep 6, 2016, at 06:34, Martin Panter wrote:
> Yes, that was basically it. Though I had only thought as far as simple
> encodings like ASCII, where one byte corresponds to one character. I
> wonder if you really need UTF-8 support. Are the encoding values
> currently encountered for Windows consoles all single-byte encodings
> or are they more complicated?

Windows supports Chinese, Japanese, and Korean encodings (code pages
936, 932, 949) that are multi-byte with one or two bytes per character.
I'm not sure how that affects this though or what your point about not
needing UTF-8 is.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Koos Zevenhoven]

On Mon, Sep 5, 2016 at 3:30 AM, Random832  wrote:
> On Sun, Sep 4, 2016, at 16:42, Koos Zevenhoven wrote:
>> On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
>> >
>> > There are two self-consistent sets of names:
>> >
>>
>> Let me add a few. I wonder if this is really used so much that
>> bytes.chr is too long to type (and you can do bchr = bytes.chr if you
>> want to):
>>
>> bytes.chr (or bchr in builtins)
>> bytes.chr_at, bytearray.chr_at
>
> Ugh, that "at" is too reminiscent of java. And it just feels wrong to
> spell it "chr" rather than "char" when there's a vowel elsewhere in the
> name.
>

Oh, I didn't realize that connection. It's funny that I get a Java
connotation from get* methods ;).


> Hmm... how offensive to the zen of python would it be to have "magic" to
> allow both bytes.chr(65) and b'ABCDE'.chr[0]? (and possibly also
> iter(b'ABCDE'.chr)? That is, a descriptor which is callable on the
> class, but returns a view on instances?

Indeed quite magical, while I really like how easy it is to remember
this *once you realize what is going on*.

I think bytes.char (on class) and data.chars (on instance) would be
quite similar.

-- Koos


> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

[Martin Panter]

On 5 September 2016 at 21:40, eryk sun  wrote:
> On Mon, Sep 5, 2016 at 7:54 PM, Steve Dower  wrote:
>> On 05Sep2016 1234, eryk sun wrote:
>>> It would probably be simpler to use UTF-16 in the main pipeline and
>>> implement Martin's suggestion to mix in a UTF-8 buffer. The UTF-16
>>> buffer could be renamed as "wbuffer", for expert use. However, if
>>> you're fully committed to transcoding in the raw layer, I'm certain
>>> that these problems can be addressed with small buffers and using
>>> Python's codec machinery for a flexible mix of "surrogatepass" and
>>> "replace" error handling.
>>
>> I don't think it actually makes things simpler. Having two buffers is
>> generally a bad idea unless they are perfectly synced, which would be
>> impossible here without data corruption (if you read half a utf-8 character
>> sequence and then read the wide buffer, do you get that character or not?).
>
> Martin's idea, as I understand it, is a UTF-8 buffer that reads from
> and writes to the text wrapper.

Yes, that was basically it. Though I had only thought as far as simple
encodings like ASCII, where one byte corresponds to one character. I
wonder if you really need UTF-8 support. Are the encoding values
currently encountered for Windows consoles all single-byte encodings
or are they more complicated?

> It necessarily consumes at least one
> character and buffers it to allow reading per byte. Likewise for
> writing, it buffers bytes until it can write a character to the text
> wrapper. ISTM, it has to look for incomplete lead-continuation byte
> sequences at the tail end, to hold them until the sequence is
> complete, at which time it either decodes to a valid character or the
> U+FFFD replacement character.

This buffering behaviour would be necessary for a multi-byte encodings
like UTF-8.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com