Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing

[Stephane Wirtel]

On 02/05, Barry Warsaw wrote:

On Feb 5, 2019, at 02:24, Stephane Wirtel  wrote:
You’re welcome!  I just pushed an update to add 3.8.0a1 to the set of Python’s 
(including git head).  Do you think there’s a better way to publicize these 
images?


I know that Julien Palard wanted a docker image with all the versions of
Python, see: https://github.com/docker-library/python/issues/373

For my part, I wanted to propose a docker image with the last version of
Python and try to use it for the detection of bugs in the main python
projects (django, numpy, flask, pandas, etc...) with a CI (example:
Gitlab-CI)

First issue: pytest uses the ast module of python and since 3.8.0a1, the
tests do not pass -> new issue for pytest

Cheers,

Stéphane

--
Stéphane Wirtel - https://wirtel.be - @matrixise
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPython on Windows ARM32

[Zachary Ware]

On Tue, Feb 5, 2019 at 7:37 PM Steve Dower  wrote:
> I also haven't reviewed the changes yet, but provided nobody is strongly
> opposed to taking on a supported platform (without additional releases
> on python.org), I expect I'll do a big part of the reviewing then.

I'm all for the first two changes (especially the second), and if 10
years of pledged corporate support for a new platform is the price we
have to pay for them, I'm ok with that :).  I expect I'll be
automatically added to any issues/PRs that come of this, but I'll keep
an eye out for them anyway and give reviews as I'm able.  I'll also
help get the build bots set up when we're ready for them.

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CPython on Windows ARM32

[Steve Dower]

Just confirming for the list that I'm aware of this and supportive, but
am not the dedicated support for this effort.

I also haven't reviewed the changes yet, but provided nobody is strongly
opposed to taking on a supported platform (without additional releases
on python.org), I expect I'll do a big part of the reviewing then.

Cheers,
Steve

On 05Feb.2019 1709, Paul Monson via Python-Dev wrote:
> Hi Python Developers,
> 
> I'm Paul Monson, I've spent about 20 years working with embedded
> software.  Since 2010 I've worked for Microsoft as a developer.
> 
> Our team is working with CPython on Azure IoT Edge devices that run on
> x64-based devices.
> 
> We would like to extend that support to Windows running on ARM 32-bit
> devices and have a working proof-of-concept.  Our team is prepared to
> provide support for CPython for Windows on ARM32 for 10 years, and to
> provide build bots for ARM32.
> 
> I like to propose that the initial sequence of PRs could be:
> - Update to OpenSSL 1.1.1 (without anything ARM specific) - ready to go
> - Migrate to libffi directly (finish 
> https://github.com/python/cpython/pull/3806)
> - Build file updates for OpenSSL ARM and check into cpython-bin-deps
> - Build file updates for CPython ARM
> - ctypes updates for ARM
> - Test module skips for ARM
> - Library updates and related test fixes for ARM
> 
> Updating OpenSSL and libffi are independent of ARM support but need to
> be done as prerequisites.  OpenSSL 1.1.0 doesn't have support for ARM32
> on Windows but OpenSSL 1.1.1 does.
> 
> I have OpenSSL 1.1.1a ready to check in to master with all tests passing
> on x86 and x64 on Windows.  Since work has already been done on this for
> other platforms only very small changes were needed for Windows.
> 
> I have also integrated and tested the current libffi on Windows x64. 
> Some additonal porting of x86 assembler to MSVC tools will need to be
> done.  I have a working port of ARM32 assembler for MSVC but it may need
> to be brought up to date and cleaned up.
> 
> The last four all need to go in together, but can be reviewed separately.
> 
> We are not planning to support Tk/Tcl on ARM32 because Windows IoT Core,
> Windows containers don't support GDI, which is a depenency of Tk/Tcl.
> 
> Since Window IoT Core and Windows container don't support the .msi or
> .exe installers found on python.org my team at Microsoft will build the
> CPython for Windows ARM32 from the official repo and distribute it.
> 
> Thanks in advance,
> 
> Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] CPython on Windows ARM32

[Paul Monson via Python-Dev]

Hi Python Developers,



I'm Paul Monson, I've spent about 20 years working with embedded software.  
Since 2010 I've worked for Microsoft as a developer.



Our team is working with CPython on Azure IoT Edge devices that run on 
x64-based devices.



We would like to extend that support to Windows running on ARM 32-bit devices 
and have a working proof-of-concept.  Our team is prepared to provide support 
for CPython for Windows on ARM32 for 10 years, and to provide build bots for 
ARM32.



I like to propose that the initial sequence of PRs could be:
- Update to OpenSSL 1.1.1 (without anything ARM specific) - ready to go
- Migrate to libffi directly (finish 
https://github.com/python/cpython/pull/3806)
- Build file updates for OpenSSL ARM and check into cpython-bin-deps
- Build file updates for CPython ARM
- ctypes updates for ARM
- Test module skips for ARM
- Library updates and related test fixes for ARM



Updating OpenSSL and libffi are independent of ARM support but need to be done 
as prerequisites.  OpenSSL 1.1.0 doesn't have support for ARM32 on Windows but 
OpenSSL 1.1.1 does.



I have OpenSSL 1.1.1a ready to check in to master with all tests passing on x86 
and x64 on Windows.  Since work has already been done on this for other 
platforms only very small changes were needed for Windows.



I have also integrated and tested the current libffi on Windows x64.  Some 
additonal porting of x86 assembler to MSVC tools will need to be done.  I have 
a working port of ARM32 assembler for MSVC but it may need to be brought up to 
date and cleaned up.



The last four all need to go in together, but can be reviewed separately.



We are not planning to support Tk/Tcl on ARM32 because Windows IoT Core, 
Windows containers don't support GDI, which is a depenency of Tk/Tcl.



Since Window IoT Core and Windows container don't support the .msi or .exe 
installers found on python.org my team at Microsoft will build the CPython for 
Windows ARM32 from the official repo and distribute it.



Thanks in advance,

Paul

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing

[Barry Warsaw]

On Feb 5, 2019, at 02:24, Stephane Wirtel  wrote:
> 
> I was not aware of this image. So it's true that it's very useful.
> 
> Thank you very much,

You’re welcome!  I just pushed an update to add 3.8.0a1 to the set of Python’s 
(including git head).  Do you think there’s a better way to publicize these 
images?

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bpo-32972: Add unittest.AsyncioTestCase review (for 3.8?)

[Terry Reedy]

On 2/5/2019 11:44 AM, Antoine Pitrou wrote:


I cannot comment on the PR, but since the functionality is
asyncio-specific, I would suggest moving it to a dedicate
`asyncio.testing` module, or something similar, rather than leaving it
in `unittest` proper.


That is one of the options discussed on the issue.

 On Tue, 5 Feb 2019 07:05:05 -0500

David Shawley  wrote:

Hi everyone, I added a PR to add a sub-class of unittest.TestCase that makes it 
possible to write async test methods.  I wrote this a few months ago and it is 
waiting on core review.  Is there a core dev that can take up this review?  I 
would love to have this functionality in the core.

Lukasz - should we add this to Python 3.8 or is it too late for feature 
additions?


Features can be added until beta1, and until that, additions are not the 
release manager decision.



BPO link: https://bugs.python.org/issue32972 

Github PR: https://github.com/python/cpython/pull/10296 



All or most of the relevant people are nosy on the issue.  So a reminder 
there would be appropriate.  However, from my cursory scan, it is not 
clear if the 5 core devs involved (marked by blue and yellow snakes) 
agree on exactly what more should be added.  Perhaps you should 
summarize what you think there is and is not agreement on.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for reversion

[Ethan Furman]

On 02/05/2019 11:35 AM, Raymond Hettinger wrote:


How about we stop using a highly public forum to pile up on Davin (being the 
subject of a thread like this can be a soul crushing experience).


Thank you for the reminder.


 Right now, he could really use some help and support from everyone on the team.


I am really looking forward to this enhancement.  Thank you, Davin, and 
everyone else who has, and will, work on it.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for reversion

[Neil Schemenauer]

I wrote:
> Could we somehow mark these APIs as experimental in 3.8?

It seems the change "e5ef45b8f519a9be9965590e1a0a587ff584c180" the
one we are discussing.  It adds two new files:

  Lib/multiprocessing/shared_memory.py
  Modules/_multiprocessing/posixshmem.c

It doesn't introduce new C APIs.  So, only
multiprocessing.shared_memory seems public.  I see we have PEP 411
that should cover this case:

  https://www.python.org/dev/peps/pep-0411/

The setup.py code could be more defensive.  Maybe only build on
platforms that have supported word sizes etc?  For 3.8, could it be
activated by uncommenting a line in Modules/Setup, rather than by
setup.py?

What happens in shared_memory if the _posixshmem module is not
available?  On Windows it seems like an import error is raised.
Otherwise, _PosixSharedMemory becomes 'object'.  Does that mean the
API still works but you lose the zero-copy speed?

Regards,

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for reversion

[Raymond Hettinger]

> On Feb 5, 2019, at 9:52 AM, Giampaolo Rodola'  wrote:
> 
>  The main problem I have with this PR is that it seems to introduce 8 brand 
> new APIs, but since there is no doc, docstrings or tests it's unclear which 
> ones are supposed to be used, how or whether they are supposed to supersede 
> or deprecate older (slower) ones involving inter process communication.

The release manger already opined that if tests and docs get finished for the 
second alpha, he prefers not to have a reversion and would rather on build on 
top of what already shipped in the first alpha.  FWIW, the absence of docs 
isn't desirable but it isn't atypical.  PEP 572 code landed without the docs. 
Docs for dataclasses arrived much after the code. The same was true for the 
decimal module. Hopefully, everyone will team up with Davin and help him get 
the ball over the goal line.

BTW, this is a feature we really want.  Our multicore story for Python isn't a 
good one.  Due to the GIL, threading usually can't exploit multiple cores for 
better performance.  Async has lower overhead than threading but achieves its 
gains by keeping all the data in a single process.  That leaves us with 
multiprocessing where the primary obstacle has been the heavy cost of moving 
data between processes.  If that cost can be reduced, we've got a winning story 
for multicore.

This patch is one of the better things that is happening to Python.  Aside from 
last week's procedural missteps and communication issues surrounding the 
commit, the many months of prior work on this have been stellar. How about we 
stop using a highly public forum to pile up on Davin (being the subject of a 
thread like this can be a soul crushing experience).  Right now, he could 
really use some help and support from everyone on the team.


Raymond


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for reversion

[Neil Schemenauer]

On 2019-02-05, Giampaolo Rodola' wrote:
> The main problem I have with this PR is that it seems to introduce
> 8 brand new APIs, but since there is no doc, docstrings or tests
> it's unclear which ones are supposed to be used, how or whether
> they are supposed to supersede or deprecate older (slower) ones
> involving inter process communication.

New or changed APIs are my major concern as well.  Localized
problems can be fixed later without much trouble.  However, APIs
"lock" us in and make it harder to change things later.  Also, will
new APIs need to be eventually supported by other Python
implementations?  I would imagine that doing zero-copy mixed with
alternative garbage collection strategies could be complicated.
Could we somehow mark these APIs as experimental in 3.8?

My gut reaction is that we shouldn't revert.  However, looking at
the changes, it seems 'multiprocessing.shared_memory' could be an
external extension package that lives in PyPI.  It doesn't require
changes to other interpreter internals.  It doesn't seem to require
internal Python header files.

Regards,

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Asking for reversion

[Giampaolo Rodola']

On Mon, Feb 4, 2019 at 4:21 AM Davin Potts <
python+python_...@discontinuity.net> wrote:

> I am attempting to do the right thing and am following the advice of other
> core devs in what I have done thus far.
>
> Borrowing heavily from what I've added to issue35813 just now:
>
> This work is the result of ~1.5 years of development effort, much of it
> accomplished at the last two core dev sprints.  The code behind it has been
> stable since September 2018 and tested as an independently installable
> package by multiple people.
>
> I was encouraged by Lukasz, Yury, and others to check in this code early,
> not waiting for tests and docs, in order to both solicit more feedback and
> provide for broader testing.  I understand that doing such a thing is not
> at all a novelty.
>

Actually it is a novelty (you should wait for review and approval). The
main problem I have with this PR is that it seems to introduce 8 brand new
APIs, but since there is no doc, docstrings or tests it's unclear which
ones are supposed to be used, how or whether they are supposed to supersede
or deprecate older (slower) ones involving inter process communication. The
introduction of new APIs in the stdlib is a sensitive topic because once
they get in they stay in, so a discussion should occur early on,
definitively not at alphaX stage. Don't mean to point fingers here, the
goal in itself (zero-copy, a topic I recently contributed to myself for the
shutil module) is certainly valuable, but I concur and think this change
should be reverted and post-poned for 3.9.

-- 
Giampaolo - http://grodola.blogspot.com
-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bpo-32972: Add unittest.AsyncioTestCase review (for 3.8?)

[Antoine Pitrou]

Hi David,

I cannot comment on the PR, but since the functionality is
asyncio-specific, I would suggest moving it to a dedicate
`asyncio.testing` module, or something similar, rather than leaving it
in `unittest` proper.

Regards

Antoine.


On Tue, 5 Feb 2019 07:05:05 -0500
David Shawley  wrote:
> Hi everyone, I added a PR to add a sub-class of unittest.TestCase that makes 
> it possible to write async test methods.  I wrote this a few months ago and 
> it is waiting on core review.  Is there a core dev that can take up this 
> review?  I would love to have this functionality in the core.
> 
> Lukasz - should we add this to Python 3.8 or is it too late for feature 
> additions?
> 
> BPO link: https://bugs.python.org/issue32972 
> 
> Github PR: https://github.com/python/cpython/pull/10296 
> 
> 
> cheers, dave.
> --
> "State and behavior. State and behavior. If it doesn’t bundle state and 
> behavior in a sensible way, it should not be an object, and there should not 
> be a class that produces it." eevee
> 
> 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] bpo-32972: Add unittest.AsyncioTestCase review (for 3.8?)

[David Shawley]

Hi everyone, I added a PR to add a sub-class of unittest.TestCase that makes it 
possible to write async test methods.  I wrote this a few months ago and it is 
waiting on core review.  Is there a core dev that can take up this review?  I 
would love to have this functionality in the core.

Lukasz - should we add this to Python 3.8 or is it too late for feature 
additions?

BPO link: https://bugs.python.org/issue32972 

Github PR: https://github.com/python/cpython/pull/10296 


cheers, dave.
--
"State and behavior. State and behavior. If it doesn’t bundle state and 
behavior in a sensible way, it should not be an object, and there should not be 
a class that produces it." eevee

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing

[Stephane Wirtel]

Hi Barry,

I was not aware of this image. So it's true that it's very useful.

Thank you very much,

Stéphane

On 02/04, Barry Warsaw wrote:

On Feb 4, 2019, at 05:02, Stephane Wirtel  wrote:


Just one idea, we could create a Docker image with this alpha version.

This Docker image could be used with the CI of the main projects and the
test suites of these projects.

If we have some issues, we should create an issue for python 3.8.0a1.


The time machine strikes again!

https://gitlab.com/python-devs/ci-images/tree/master

We call these “semi-official”!  The current image takes a slightly different 
approach, by including all the latest Python versions from 2.7, and 3.4-3.8, 
plus git head.  I just pushed an update for the latest Python 3.8 alpha and 
3.7.2.  It’s building now, but the image should be published on quay.io as soon 
as that’s done.

Contributions most welcome!

-Barry






___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stephane%40wirtel.be



--
Stéphane Wirtel - https://wirtel.be - @matrixise
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com