[Python-Dev] [RELEASE] Python 3.7.0b2 is now available for testing

[Ned Deily]

On behalf of the Python development community and the Python 3.7 release
team, I'm happy to announce the availability of Python 3.7.0b2.  b2 is
the second of four planned beta releases of Python 3.7, the next major
release of Python, and marks the end of the feature development phase
for 3.7.  You can find Python 3.7.0b2 here:

https://www.python.org/downloads/release/python-370b2/

Among the new major new features in Python 3.7 are:

* PEP 538, Coercing the legacy C locale to a UTF-8 based locale
* PEP 539, A New C-API for Thread-Local Storage in CPython
* PEP 540, UTF-8 mode
* PEP 552, Deterministic pyc
* PEP 553, Built-in breakpoint()
* PEP 557, Data Classes
* PEP 560, Core support for typing module and generic types
* PEP 562, Module __getattr__ and __dir__
* PEP 563, Postponed Evaluation of Annotations
* PEP 564, Time functions with nanosecond resolution
* PEP 565, Show DeprecationWarning in __main__
* PEP 567, Context Variables

Please see "What’s New In Python 3.7" for more information.
Additional documentation for these features and for other changes
will be provided during the beta phase.

https://docs.python.org/3.7/whatsnew/3.7.html

Beta releases are intended to give you the opportunity to test new
features and bug fixes and to prepare their projects to support the
new feature release. We strongly encourage you to test your projects
with 3.7 during the beta phase and report issues found to
https://bugs.python.org as soon as possible.

While the release is feature complete entering the beta phase, it is
possible that features may be modified or, in rare cases, deleted up
until the start of the release candidate phase (2018-05-21). Our goal
is have no ABI changes after beta 3 and no code changes after rc1.
To achieve that, it will be extremely important to get as much exposure
for 3.7 as possible during the beta phase.

Attention macOS users: as of b1, there is a new installer variant for
macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. This
variant is expected to become the default version when 3.7.0 releases.
Check it out! We welcome your feedback.

Please keep in mind that this is a preview release and its use is
not recommended for production environments.

The next planned release of Python 3.7 will be 3.7.0b3, currently
scheduled for 2018-03-26. More information about the release schedule
can be found here:

https://www.python.org/dev/peps/pep-0537/


--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the dataclass frozen property apply to subclasses?

[Eric V. Smith]

On 2/22/2018 1:56 AM, Raymond Hettinger wrote:

When working on the docs for dataclasses, something unexpected came up.  If a 
dataclass is specified to be frozen, that characteristic is inherited by 
subclasses which prevents them from assigning additional attributes:

 >>> @dataclass(frozen=True)
 class D:
 x: int = 10

 >>> class S(D):
 pass

 >>> s = S()
 >>> s.cached = True
 Traceback (most recent call last):
   File "", line 1, in 
 s.cached = True
   File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dataclasses.py",
 line 448, in _frozen_setattr
 raise FrozenInstanceError(f'cannot assign to field {name!r}')
 dataclasses.FrozenInstanceError: cannot assign to field 'cached'

Other immutable classes in Python don't behave the same way:


 >>> class T(tuple):
 pass

 >>> t = T([10, 20, 30])
 >>> t.cached = True

 >>> class F(frozenset):
 pass

 >>> f = F([10, 20, 30])
 >>> f.cached = True

 >>> class B(bytes):
 pass

 >>> b = B()
 >>> b.cached = True



I'll provide some background, then get in to the dataclasses design 
issues. Note that I'm using "field" here in the PEP 557 sense.


There are some questions to resolve:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass?


2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass?


3. What happens when a non-dataclass inherits from a frozen dataclass?

4. Can new non-field attributes be created for frozen dataclasses?


I think it's useful to look at what attrs does. Unsurprisingly, attrs 
works the way the dataclasses implementation in 3.7.0a1 works:


- If a frozen attrs class inherits from a non-frozen attrs class, the 
result is a frozen attrs class.


- If a non-frozen attrs class inherits from a frozen attrs class, the 
result is a frozen attrs class.


- For a frozen attrs class, you may not assign to any field, nor create 
new non-field instance attributes.


- If a non-attrs class derives from a frozen attrs class, then you 
cannot assign to or create any non-field instance attributes. This is 
because they override the class's __setattr__ to always raise. This is 
the case that Raymond initially brought up on this thread (but for 
dataclasses, of course).


As I said, this is also how 3.7.0a1 dataclasses also works. The only 
difference between this and 3.7.0.a2 is that I prohibited inheriting a 
non-frozen dataclass from a frozen one, and also prohibited the 
opposite: you can't inherit a frozen dataclass from a non-frozen 
dataclass. This was just a stop-gap measure to give us more wiggle room 
for future changes. But this does nothing to address Raymond's concern 
about non-dataclasses deriving from frozen dataclasses.


A last piece of background info on how dataclasses and attrs work: the 
most derived class implements all of the functionality. They never call 
in to the base class to do anything. The base classes just exist to 
provide the list of fields.


If frozen dataclasses only exist to protect fields that belong to the 
hash, then my suggestion is to change the implementation of frozen class 
to use properties instead of overwriting __setattr__ (Nick also 
suggested this). This would allow you to create non-field attributes. If 
the purpose is really to prevent any attributes from being added or 
modified, then I think __setattr__ should stay but we should change it 
to allow non-dataclass subclasses to add non-field instance attributes 
(addressing Raymond's concern).


I think we shouldn't allow non-field instance attributes to be added to 
a frozen dataclass, although if anyone has a strong feeling about it, 
I'd like to hear it.


So, given a frozen dataclass "C" with field names in "field_names", I 
propose changing __setattr__ to be:


def __setattr__(self, name, value):
if type(self) is C or name in field_names:
raise FrozenInstanceError(f'cannot assign to field {name!r}')
super(cls, self).__setattr__(name, value)

In the current 3.7.0a2 implementation of frozen dataclasses, __setattr__ 
always raises. The change is the test and then call to 
super().__setattr__ if it's a derived class. The result is an exception 
if either self is an instance of C, or if self is an instance of a 
derived class, but the attribute being set is a field of C.


So going back to original questions above, my suggestions are:

1. What happens when a frozen dataclass inherits from a non-frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior.


2. What happens when a non-frozen dataclass inherits from a frozen 
dataclass? The result is a frozen dataclass, and all fields are 
non-writable. No non-fields can be added. This is a reversion to the 
3.7.0a1 behavior. I'd also be okay with this case being an 

Re: [Python-Dev] Backward incompatible change about docstring AST

[Brett Cannon]

On Tue, 27 Feb 2018 at 05:38 INADA Naoki  wrote:

> Hi, all.
>
> There is design discussion which is deferred blocker of 3.7.
> https://bugs.python.org/issue32911
>
> ## Background
>
> An year ago, I moved docstring in AST from statements list to field of
> module, class and functions.
> https://bugs.python.org/issue29463
>
> Without this change, AST-level constant folding was complicated because
> "foo" can be docstring but "fo" + "o" can't be docstring.
>
> This simplified some other edge cases.  For example, future import must
> be on top of the module, but docstring can be before it.
> Docstring is very special than other expressions/statement.
>
> Of course, this change was backward incompatible.
> Tools reading/writing docstring via AST will be broken by this change.
> For example, it broke PyFlakes, and PyFlakes solved it already.
>
> https://github.com/PyCQA/pyflakes/pull/273
>
> Since AST doesn't guarantee backward compatibility, we can change
> AST if it's reasonable.
>
> Last week, Mark Shannon reported issue about this backward incompatibility.
> As he said, this change losted lineno and column of docstring from AST.
>
> https://bugs.python.org/issue32911#msg312567
>
>
> ## Design discussion
>
> And as he said, there are three options:
>
> https://bugs.python.org/issue32911#msg312625
>
> > It seems to be that there are three reasonable choices:
> > 1. Revert to 3.6 behaviour, with the addition of `docstring` attribute.
> > 2. Change the docstring attribute to an AST node, possibly by modifying
> the grammar.
> > 3. Do nothing.
>

I'm +1 for #3, +0 for #2, -1 for #1.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backward incompatible change about docstring AST

[Serhiy Storchaka]

28.02.18 00:31, Terry Reedy пише:
2. Usually the position of the docstring is used for determining the 
absolute position of some fragments in the docstring (for example 
doctests). But since the literal string can contain \n and escaped 
newlines, and this information is lost in AST, the position of the 
fragment can be determined accurately.


You obviously meant 'cannot be determined accurately', because for 
reasons including the ones you gave, the relation between string 
literals in code and the resulting string objects is many-to-one, (as 
with int literals).


Yes, thank you for correction.

The reasons are that the relation between lines of source code and lines 
of the resulting string objects is not easy. The string literal 'a\nb\c' 
takes one line in the source code, but produces a three-line string 
object. On other side, the string literal


'''\
abc\
'''

takes three lines in the source code, but produces a single-line string 
object. And it is not so rare that a docstring starts with an escaped 
newline.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backward incompatible change about docstring AST

[Terry Reedy]

On 2/27/2018 9:32 AM, Serhiy Storchaka wrote:

27.02.18 15:37, INADA Naoki пише:

Of course, this change was backward incompatible.
Tools reading/writing docstring via AST will be broken by this change.
For example, it broke PyFlakes, and PyFlakes solved it already.

https://github.com/PyCQA/pyflakes/pull/273


Other examples:

coveragepy: https://bitbucket.org/ned/coveragepy/commits/99176232199b
pytest: https://github.com/pytest-dev/pytest/pull/2870


Last week, Mark Shannon reported issue about this backward 
incompatibility.

As he said, this change losted lineno and column of docstring from AST.


While losing lineno and column is a loss, it is not so large. There are 
existing issues with docstring position.


1. CPython and PyPy set different position for multiline strings. PyPy 
sets the position of the start of string, but CPython sets the position 
of the end of the string. A program that utilizes the docstring position 
needs to handle both of these cases.


2. Usually the position of the docstring is used for determining the 
absolute position of some fragments in the docstring (for example 
doctests). But since the literal string can contain \n and escaped 
newlines, and this information is lost in AST, the position of the 
fragment can be determined accurately.


You obviously meant 'cannot be determined accurately', because for 
reasons including the ones you gave, the relation between string 
literals in code and the resulting string objects is many-to-one, (as 
with int literals).



This is just a best effort approximation.

3. You can determine an approximate position of the docstring by 
positions of preceding or following nodes.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

[Alexander Belopolsky via Python-Dev]

On Mon, Feb 26, 2018 at 7:51 PM, Guido van Rossum  wrote:
..
> The reason is that for people who are not Python experts there's no obvious
> reason why `for VAR = EXPR` should mean one thing and `for VAR in EXPR`
> should mean another.

This would be particularly surprising for people exposed to Julia
where these two forms are equivalent:

julia> for x = [1,2] println(x); end
1
2

julia> for x in [1,2] println(x); end
1
2
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467 (Minor API improvements for binary sequences) - any thoughts?

[Ethan Furman]

On 02/26/2018 11:34 PM, Elias Zamaria wrote:


Nick, I'm trying to reply to your message, but I can't figure out how.

You mentioned that the PEP needs a "champion". What would that involve? How 
much time and effort would it take? What
kinds of decisions would I make?


Being a PEP "champion" involves collecting lots of data, sorting it, making decisions about API design, posting about 
those decisions along with the pros and cons, listening to more feedback, and continuing until there is general 
agreement about the PEP.  After that a request is made to (usually) Guido to accept or reject the PEP.  If accepted, 
then the code writing stage happens.


Writing code first is not bad as a working proof-of-concept is always handy.


The iterbytes thing in the PEP is something I was wishing for, while working on 
a personal project. I stumbled upon this
PEP and decided to try to implement it myself, to learn about C and the Python 
internals, among other reasons.


It's a good way to go about it!


I don't know how I would feel working on something so general, of use to so 
many people for lots of different purposes.
Do I know enough about all of the use cases and what everyone wants? I am not 
completely against it but I'd need to
think about it.


Part of the PEP writing process is asking for and collecting use-cases;  if possible, looking at other code projects for 
use-cases is also useful.


Time needed can vary widely depending on the subject; if I recall correctly, PEP 409 only took a few days, while PEP 435 
took several weeks.  PEP 467 has already gone through a few iterations, so hopefully not too much more time is required.


If you would like to try you'll get plenty of help from the community -- at least from those willing to go through PEP 
467 again.  ;)


At the moment, though, we're concentrating on getting v3.7.0 as bug-free as possible, so feel free to research where the 
PEP is now and go through the last discussion, but you should wait for the 3.7.0 release before actively bringing the 
PEP discussion back to python-dev.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467 (Minor API improvements for binary sequences) - any thoughts?

[Elias Zamaria]

Nick, I'm trying to reply to your message, but I can't figure out how.

You mentioned that the PEP needs a "champion". What would that involve? How
much time and effort would it take? What kinds of decisions would I make?

The iterbytes thing in the PEP is something I was wishing for, while
working on a personal project. I stumbled upon this PEP and decided to try
to implement it myself, to learn about C and the Python internals, among
other reasons.

I don't know how I would feel working on something so general, of use to so
many people for lots of different purposes. Do I know enough about all of
the use cases and what everyone wants? I am not completely against it but
I'd need to think about it.

On Wed, Feb 21, 2018 at 2:36 PM, Chris Barker  wrote:

>
>
> On Wed, Feb 21, 2018 at 12:39 PM, Steve Holden 
> wrote:
>
>> I think the chances of a "byte" object are about as good as the chances
>> of a character object
>>
>
> probably right.
>
>
>> (though one can always implement such in C extensions, that wouldn't
>> build them into the syntax).
>>
>
> I think you could simply subclass, too (overriding __new__ and a couple
> methods). But that would do exactly no good, unless you used your own
> custom string and bytes objects, too. The whole point is that iterating
> over a string (Or bytes) always returns an also-iterable object,
> ad-infinitum.
>
> This is the cause of the major remaining common "type error" in Python.
> (the old integer division used to be the big one)
>
>
>> The fact that characters are single-byte strings is responsible for
>> certain anomalies with (e.g.) the __contains__ operator (list elements
>> aren't lists, but string element are strings), but overall the choices made
>> lead to sensible, comprehensible code.
>>
>
> I'm pretty convinced that the choice not to have a character type has had
> basically zero benefits to sensible, comprehensible code, though it's not
> a very big deal, either. not a big enough deal for the churn it would cause
> to introduce it now, that's for sure.
>
> so +1 for this PEP as is.
>
> -CHB
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE
> 
>   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Disable built-in module import in embedded Python

[Viktor Kovacs]

Hi All,

I have a question about embedding python in C++. I don't want to create a
duplication so I just leave here the existing stack overflow question:
https://stackoverflow.com/questions/48992030/disable-built-in-module-import-in-embedded-python

Do you have any idea how to achieve this?

Thanks,
Viktor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backward incompatible change about docstring AST

[Serhiy Storchaka]

27.02.18 15:37, INADA Naoki пише:

Of course, this change was backward incompatible.
Tools reading/writing docstring via AST will be broken by this change.
For example, it broke PyFlakes, and PyFlakes solved it already.

https://github.com/PyCQA/pyflakes/pull/273


Other examples:

coveragepy: https://bitbucket.org/ned/coveragepy/commits/99176232199b
pytest: https://github.com/pytest-dev/pytest/pull/2870



Last week, Mark Shannon reported issue about this backward incompatibility.
As he said, this change losted lineno and column of docstring from AST.


While losing lineno and column is a loss, it is not so large. There are 
existing issues with docstring position.


1. CPython and PyPy set different position for multiline strings. PyPy 
sets the position of the start of string, but CPython sets the position 
of the end of the string. A program that utilizes the docstring position 
needs to handle both of these cases.


2. Usually the position of the docstring is used for determining the 
absolute position of some fragments in the docstring (for example 
doctests). But since the literal string can contain \n and escaped 
newlines, and this information is lost in AST, the position of the 
fragment can be determined accurately. This is just a best effort 
approximation.


3. You can determine an approximate position of the docstring by 
positions of preceding or following nodes.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Backward incompatible change about docstring AST

[INADA Naoki]

Hi, all.

There is design discussion which is deferred blocker of 3.7.
https://bugs.python.org/issue32911

## Background

An year ago, I moved docstring in AST from statements list to field of
module, class and functions.
https://bugs.python.org/issue29463

Without this change, AST-level constant folding was complicated because
"foo" can be docstring but "fo" + "o" can't be docstring.

This simplified some other edge cases.  For example, future import must
be on top of the module, but docstring can be before it.
Docstring is very special than other expressions/statement.

Of course, this change was backward incompatible.
Tools reading/writing docstring via AST will be broken by this change.
For example, it broke PyFlakes, and PyFlakes solved it already.

https://github.com/PyCQA/pyflakes/pull/273

Since AST doesn't guarantee backward compatibility, we can change
AST if it's reasonable.

Last week, Mark Shannon reported issue about this backward incompatibility.
As he said, this change losted lineno and column of docstring from AST.

https://bugs.python.org/issue32911#msg312567


## Design discussion

And as he said, there are three options:

https://bugs.python.org/issue32911#msg312625

> It seems to be that there are three reasonable choices:
> 1. Revert to 3.6 behaviour, with the addition of `docstring` attribute.
> 2. Change the docstring attribute to an AST node, possibly by modifying the 
> grammar.
> 3. Do nothing.

1 is backward compatible about reading docstring.
But when writing, it's not DRY or SSOT.  There are two source of docstring.
For example: `ast.Module([ast.Str("spam")], docstring="egg")`

2 is considerable.  I tried to implement this idea by adding `DocString`
statement AST.
https://github.com/python/cpython/pull/5927/files

While it seems large change, most changes are reverting the AST changes.
So it's more closer to 3.6 codebase.  (especially, test_ast is very
close to 3.6)

In this PR, `ast.Module([ast.Str("spam")])` doesn't have docstring for
simplicity.  So it's backward incompatible for both of reading and
writing docstring too.
But it keeps lineno and column of docstring in AST.

3 is most conservative because 3.7b2 was cut now and there are some tools
supporting 3.7 already.


I prefer 2 or 3.  If we took 3, I don't want to do 2 in 3.8.  One
backward incompatible
change is better than two.

Any thoughts?

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How is the GitHub workflow working for people?

[Giampaolo Rodola']

I personally use a GIT commit hook which runs flake8 against the *modified
py files only* and rejects the commit in case of non-compliance:
https://github.com/giampaolo/psutil/blob/master/.git-pre-commit
...I install it via make:
https://github.com/giampaolo/psutil/blob/ad4acae5489f86fc3bef645505b3873f156b4867/Makefile#L186
...and once the whole code base is in a good shape have Travis run flake8
against the whole code base on each commit.

When I join an existing project I try to enforce the same workflow and rely
on pep8ify (https://github.com/spulec/pep8ify) to adjust the existing code.
In order to not be too "strict" I may modify the GIT hook to emit a warning
instead of rejecting the commit.

On one hand I think it would be great to do the same for cPython. On the
other hand I see 2 big downsides:

- losing "git blame" history for A LOT of lines of code
- even if you're familiar with PEP8 it's sometimes hard to spot
non-compliant lines unless you integrate flake8 into your IDE (I do for
Sublime); putting such a constraint on contributing users is probably too
much and may discourage contributions.


On Sun, Feb 25, 2018 at 6:13 AM, Guido van Rossum  wrote:

> It's easy to only analyze the files in the diff (these linters don't do
> cross-file analysis anyways, typically) and it's possible to write a filter
> that only keeps warnings about lines that are changed, but I don't know of
> a standard solution for the latter (places where I worked where I've seen
> this always had their own custom implementation).
>
> On Sat, Feb 24, 2018 at 7:35 PM, Mariatta Wijaya <
> mariatta.wij...@gmail.com> wrote:
>
>> Can any of these said linters analyze only the diff in the PR, instead of
>> the entire CPython codebase?
>>
>> Mariatta Wijaya
>>
>> ᐧ
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
>> 40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/g.
> rodola%40gmail.com
>
>


-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com