Re: [Python-Dev] Committing a bug fix

[Alexander Belopolsky]

On Mon, Sep 28, 2015 at 4:13 AM, Terry Reedy  wrote:

> Normal one specified in devguide: commit 3.4, merge 3.5, merge 3.6,


That's exactly what I did at fist, but apparently while I was doing this,
another commit was pushed to all three branches.   To recover, I did a
series of hg update 3.x; hg merge dances fighting with Misc/NEWS on every
turn.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Brett Cannon]

See http://bugs.python.org/issue18967 if you want to join the discussion on
how to solve Misc/NEWS potentially in the short term (long term we have a
plan once someone has time to hack on the issue tracker).

On Mon, 28 Sep 2015 at 10:12 Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

>
> On Mon, Sep 28, 2015 at 4:13 AM, Terry Reedy  wrote:
>
>> Normal one specified in devguide: commit 3.4, merge 3.5, merge 3.6,
>
>
> That's exactly what I did at fist, but apparently while I was doing this,
> another commit was pushed to all three branches.   To recover, I did a
> series of hg update 3.x; hg merge dances fighting with Misc/NEWS on every
> turn.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Erik Bray]

On Fri, Sep 25, 2015 at 6:27 PM, Chris Barker - NOAA Federal
 wrote:
>
> You can use "Windows SDK for Windows 7 and .NET Framework 4".
>
> http://www.microsoft.com/en-us/download/details.aspx?id=8279
>
>
> Thanks. Last time I tried that route, it was for 64 bit py2.7. And it
> required some kludging of environment variables, and registry acces I don't
> have permission for.
>
> But it still may be the best bet. I'll give it a try when I have a chance.
>
> And this should be in the "official" docs...

For what it's worth, I've had good luck compiling *most* extension
modules on Windows using the gcc from MinGW-w64.

The big notable exception was that last time I tried compiling Numpy
with it I got a segfaulting Numpy.  But I never had a chance to
investigate why or if it's fixable.  My own extension modules work
fine on Windows when compiled in MinGW though.

Erik B.


> On Sat, Sep 26, 2015 at 12:24 AM, Chris Barker - NOAA Federal
>  wrote:
>>
>> As I understand it, the MS VS2010 compiler is required (or at least
>> best practice) for compiling Python extensions for the python.org
>> Windows builds of py 3.4 and ?[1]
>>
>> However, MS now makes it very hard (impossible?) to download VS2010
>> Express ( or Community, or whatever the free as in beer version is
>> called).
>>
>> I realize that this is not python-dev's responsibility, but if there
>> is any way to either document where it can be found, or put a bit of
>> pressure on MS to make it available, as they have for VS2008 and
>> py2.7, that would be great.
>>
>> Sorry to bug this list, I didn't know where else to reach out to.
>>
>> -Chris
>>
>> [1] it's actually prefer hard to find out which compiler version is
>> used for which python version. And has been for years. Would a patch
>> to the docs, probably here:
>>
>> https://docs.python.org/3.4/using/windows.html#compiling-python-on-windows
>>
>> Be considered?
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
>
>
>
>
> --
> INADA Naoki  
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/erik.m.bray%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Nathaniel Smith]

On Sep 28, 2015 1:39 PM, "Paul Moore"  wrote:
>
> On 28 September 2015 at 21:18, MRAB  wrote:
> > Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
> > both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
> > that wasn't due to a bug in the source code.
>
> Interesting. With Python 3.5, what CRT does the module use? With
> Python 3.4 and earlier, distutils arranged for mingw builds to link to
> the CRT that matched the CPython build,

This doesn't sound right to me. Maybe if you're talking about the original
mingw, but that's been abandoned upstream for years. Nowadays everyone uses
the mingw-w64 fork, for both 32 and 64 bits. And in mingw-w64, the only way
to select a non-default CRT is to hack up the toolchain's source tree and
cross your fingers. It's definitely not a supported configuration upstream.
(Source: we've been talking to mingw-w64 upstream and MS about how to
potentially make building free software less painful on Windows.)

> but I don't think that
> happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
> have a CRT that doesn't match the core. Obviously this isn't causing
> you issues, but I'm not sure if it could (CRT problems seem to be
> mostly about "might cause issues" type problems...)

CRT issues are indeed tricky, because they only bite in certain
circumstances -- so long as you never pass a FILE* or a fileno across the
dll boundary, or call malloc in one module and free in the other, or ...
then it works fine. Except when it doesn't :-).

Example of a subtle issue: msvc and mingw-w64 disagree about whether the
x87 fpu should be configured in extended precision mode or not. So simply
loading a mingw-w64-compiled dll can potentially cause math functions in
other dlls to start returning incorrect results. Or maybe not -- it depends
on the details of how MS implemented things, and we can't exactly check. Or
whether you get correct math result in mingw-w64 compiled code can depend
on which runtime was used to spawn the thread that's running the code (!!).
And even if things work today, it could change without warning in any
update, because no one is testing this configuration upstream.

It's totally plausible that the regex module happens to avoid all these
issues. But you can't assume that it's a general solution.

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[MRAB]

On 2015-09-28 19:00, Erik Bray wrote:

On Fri, Sep 25, 2015 at 6:27 PM, Chris Barker - NOAA Federal
 wrote:


You can use "Windows SDK for Windows 7 and .NET Framework 4".

http://www.microsoft.com/en-us/download/details.aspx?id=8279


Thanks. Last time I tried that route, it was for 64 bit py2.7. And it
required some kludging of environment variables, and registry acces I don't
have permission for.

But it still may be the best bet. I'll give it a try when I have a chance.

And this should be in the "official" docs...


For what it's worth, I've had good luck compiling *most* extension
modules on Windows using the gcc from MinGW-w64.

The big notable exception was that last time I tried compiling Numpy
with it I got a segfaulting Numpy.  But I never had a chance to
investigate why or if it's fixable.  My own extension modules work
fine on Windows when compiled in MinGW though.

Erik B.


Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
that wasn't due to a bug in the source code.



On Sat, Sep 26, 2015 at 12:24 AM, Chris Barker - NOAA Federal
 wrote:


As I understand it, the MS VS2010 compiler is required (or at least
best practice) for compiling Python extensions for the python.org
Windows builds of py 3.4 and ?[1]

However, MS now makes it very hard (impossible?) to download VS2010
Express ( or Community, or whatever the free as in beer version is
called).

I realize that this is not python-dev's responsibility, but if there
is any way to either document where it can be found, or put a bit of
pressure on MS to make it available, as they have for VS2008 and
py2.7, that would be great.

Sorry to bug this list, I didn't know where else to reach out to.

-Chris

[1] it's actually prefer hard to find out which compiler version is
used for which python version. And has been for years. Would a patch
to the docs, probably here:

https://docs.python.org/3.4/using/windows.html#compiling-python-on-windows

Be considered?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com





--
INADA Naoki  


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/erik.m.bray%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/python%40mrabarnett.plus.com




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[MRAB]

On 2015-09-28 21:38, Paul Moore wrote:

On 28 September 2015 at 21:18, MRAB  wrote:
> Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
> both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
> that wasn't due to a bug in the source code.

Interesting. With Python 3.5, what CRT does the module use? With
Python 3.4 and earlier, distutils arranged for mingw builds to link to
the CRT that matched the CPython build, but I don't think that
happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
have a CRT that doesn't match the core. Obviously this isn't causing
you issues, but I'm not sure if it could (CRT problems seem to be
mostly about "might cause issues" type problems...)


I can't remember the details off-hand, but here's the issue:

http://bugs.python.org/issue24385

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Paul Moore]

On 28 September 2015 at 21:18, MRAB  wrote:
> Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
> both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
> that wasn't due to a bug in the source code.

Interesting. With Python 3.5, what CRT does the module use? With
Python 3.4 and earlier, distutils arranged for mingw builds to link to
the CRT that matched the CPython build, but I don't think that
happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
have a CRT that doesn't match the core. Obviously this isn't causing
you issues, but I'm not sure if it could (CRT problems seem to be
mostly about "might cause issues" type problems...)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Guido van Rossum]

Your comments make total sense -- we're just short on people who can write
that kind of docs. :-(

On Mon, Sep 28, 2015 at 2:15 PM, Barry Warsaw  wrote:

> On Sep 28, 2015, at 08:22 AM, Guido van Rossum wrote:
>
> >I saw that you had a need for an asyncio tutorial. I wonder if the "500
> >lines" chapter on asyncio would help? I didn't write it; I only write the
> >500 lines of code, A. Jesse Jiryu Davis wrote the text, and it's
> wonderful:
> >http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html
>
> It's a great article, and I particularly like how it starts from first
> principles and builds essentially the asyncio framework up from them, with
> nice explorations on how things like generators work.
>
> But it's not quite what I'm looking for.  When I go to the stdlib asyncio
> docs
> page, I'm immediately thrown into (albeit good) reference documentation.  I
> though that the page "Develop with asyncio" might give me what I want, but
> it's really not.
>
> What I'm looking for is a gentle introduction, assuming that the reader
> either
> understands the basics of asynchronous i/o or maybe comes from a
> background in
> threaded programming.  It would then explain things like "this is a Future
> and
> this is what you use it for", "this is a Task and this is when you use a
> Task
> and when you use a Future", "this is when you want to define a Protocol,
> and
> this is how it hooks into the framework".  I think that kind of high-level
> overview of the classes and how they hang together is what I think is
> missing.
>
> Take for example 18.5.3 Tasks and coroutines.  Aside from the fact that the
> title would imply it would start explaining Tasks, it actually doesn't get
> to
> them until way later on the page!
>
> What's a Future?  The first thing the asyncio page says is that the "class
> is
> almost compatible with concurrent.futures.Future".  Okay, but that still
> doesn't tell me what it *is* or what it's used for.  Following the link to
> that other class isn't really enlightening. ;)
>
> I hope my comments make sense.  I know that when you already understand the
> concepts, it's difficult to distill them into the right documentation for
> very
> different audiences.  What's there seems mostly great for reference, and
> maybe
> the stdlib docs aren't the right place for what I want, but as the 500
> Lines
> chapter promotes, asyncio is pretty powerful so it would be nice to have
> Python documentation for newcomers.
>
> I'm sure I'm not the right person to write it though. ;)
>
> Cheers,
> -Barry
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Paul Moore]

On 28 September 2015 at 22:18, Nathaniel Smith  wrote:
> On Sep 28, 2015 1:39 PM, "Paul Moore"  wrote:
>>
>> On 28 September 2015 at 21:18, MRAB  wrote:
>> > Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
>> > both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
>> > that wasn't due to a bug in the source code.
>>
>> Interesting. With Python 3.5, what CRT does the module use? With
>> Python 3.4 and earlier, distutils arranged for mingw builds to link to
>> the CRT that matched the CPython build,
>
> This doesn't sound right to me. Maybe if you're talking about the original
> mingw, but that's been abandoned upstream for years. Nowadays everyone uses
> the mingw-w64 fork, for both 32 and 64 bits. And in mingw-w64, the only way
> to select a non-default CRT is to hack up the toolchain's source tree and
> cross your fingers. It's definitely not a supported configuration upstream.
> (Source: we've been talking to mingw-w64 upstream and MS about how to
> potentially make building free software less painful on Windows.)

Interesting, it's been a long time since I've done work on the mingw
support, so I hadn't realised this had happened.

I guess that means that nowadays all mingw-compiled extensions link to
msvcrt (I assume that's still the CRT that mingw-w64 uses by default)?

>> but I don't think that
>> happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
>> have a CRT that doesn't match the core. Obviously this isn't causing
>> you issues, but I'm not sure if it could (CRT problems seem to be
>> mostly about "might cause issues" type problems...)
>
> CRT issues are indeed tricky, because they only bite in certain
> circumstances -- so long as you never pass a FILE* or a fileno across the
> dll boundary, or call malloc in one module and free in the other, or ...
> then it works fine. Except when it doesn't :-).
>
> Example of a subtle issue: msvc and mingw-w64 disagree about whether the x87
> fpu should be configured in extended precision mode or not. So simply
> loading a mingw-w64-compiled dll can potentially cause math functions in
> other dlls to start returning incorrect results. Or maybe not -- it depends
> on the details of how MS implemented things, and we can't exactly check. Or
> whether you get correct math result in mingw-w64 compiled code can depend on
> which runtime was used to spawn the thread that's running the code (!!). And
> even if things work today, it could change without warning in any update,
> because no one is testing this configuration upstream.
>
> It's totally plausible that the regex module happens to avoid all these
> issues. But you can't assume that it's a general solution.

Agreed. That was essentially what I was asking - has someone found a
solution to the issue of avoiding mixed CRTs when using mingw as a
compiler (or at least, found a better solution than using the original
mingw -lmsvcr100 approach combined with the distutils code that
supports this). Looks like the answer is that nobody has. But it's
certainly (still) true that for straightforward cases things seem
pretty much OK, especially if you avoid the "obvious" issues. (The
non-obvious ones sound terrifying, and I'm glad I don't have to write
code that encounters those!!!)

Cheers,
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Guido van Rossum]

Cool! Glad to see that the idea of local core-dev sprints hasn't totally
been forgotten.

I saw that you had a need for an asyncio tutorial. I wonder if the "500
lines" chapter on asyncio would help? I didn't write it; I only write the
500 lines of code, A. Jesse Jiryu Davis wrote the text, and it's wonderful:
http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html

On Mon, Sep 28, 2015 at 7:32 AM, Barry Warsaw  wrote:

> Yesterday (27-Sep-2015), four Washington DC area Python developers got
> together for a local Python hacking sprint.  Andrew, Eric, Jason, and I
> were
> joined remotely by the esteemed R. David Murray.  We did stuff and had fun!
> We hope to do stuff and have fun again!  If you want to join us for stuff
> and
> fun next time, please get in touch.
>
> Here's a write-up of the sprint:
>
> https://gitlab.com/python-smtpd-hackers/aiosmtpd/wikis/2015-09-27-hackfest
>
> Special thanks to the PSF and Brian Curtin for sponsoring the sprint.
>
> Cheers,
> -Barry
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Washington DC Python sprint report

[Barry Warsaw]

Yesterday (27-Sep-2015), four Washington DC area Python developers got
together for a local Python hacking sprint.  Andrew, Eric, Jason, and I were
joined remotely by the esteemed R. David Murray.  We did stuff and had fun!
We hope to do stuff and have fun again!  If you want to join us for stuff and
fun next time, please get in touch.

Here's a write-up of the sprint:

https://gitlab.com/python-smtpd-hackers/aiosmtpd/wikis/2015-09-27-hackfest

Special thanks to the PSF and Brian Curtin for sponsoring the sprint.

Cheers,
-Barry


pgp4qaLhPZXB9.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Brian Ray]

I would like to propose a mixed core-dev none-core-dev sprint to occur in
Chicago within the next couple months. ChiPy (http://chipy.org) can help
sponsor. I am going to share my thoughts on this with the group-organizers <
group-organiz...@python.org> list first in order to get some feedback. Post
found here:
https://mail.python.org/pipermail/group-organizers/2015-September/000441.htm


Regards,

Brian Ray
ChiPy organizer
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Brian Ray]

meant:
https://mail.python.org/pipermail/group-organizers/2015-September/000441.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Martin Panter]

On 28/09/2015, Terry Reedy  wrote:
> You neglected to forward merge a 3.4-3.4 merge and a 3.5-3.5 merge.  I
> failed to notice the first and added a commit, and then backed it out,
> when I could not merge forward due to merge conflicts with your commits.
>   Perhaps you just need to null-merge the 3.4 commits into 3.5, but I do
> not really know how to make the repository usable again.

Hopefully I have fixed this for you. My technique is below. I first
tried with Mercurial 2.7, but updated to 3.5 which apparently grew
some support for “criss-cross” merges, and this helped a lot with the
second merge.

hg update 3.5
hg merge 3.4  # NEWS and Argument Clinic files conflict
hg revert -r . Misc/NEWS Modules/clinic/_datetimemodule.c.h
python Tools/clinic/clinic.py --make
# Verify both Alexander and Serhiy’s entries already in NEWS
hg diff -p  # No file changes to 3.5 branch
hg resolve -m
hg commit -m "Merge from 3.4 into 3.5; no file changes"
hg update default
hg merge 3.5  # No conflicts; 3 × “ambiguous merge - picked m action”
hg diff -p  # Duplicate NEWS entry
hg revert -r . Misc/NEWS
hg diff -p  # No changes
hg commit -m "Merge from 3.5; no file changes"

Summary of what I think was going on: Serhiy made some changes quite a
few hours ago to do with readline.set_completer_delims(). Meanwhile,
Alexander made some tzinfo.fromutc() commits based on revisions before
Serhiy’s changes. A couple comments on Alexander’s commits:

ff68705c56a8: Initial 3.4 patch has a trailing space in NEWS
c706c062f545: Initial merge into 3.5 removes this trailing space
cbcf82f92c25: Looks like a second version of the original patch
(ff68705c56a8), except without the extra space. I guess you got
confused, but it would have been better to not push this commit :)

Also @Terry, not sure on your workflow, but you might have been able
to avoid pushing your 3.4 commits. You might have been able to remove
it locally by using “hg strip” or deleting your local repository. Or
just leave it locally until the other branches got sorted out. Notice
that the 3.4 branch still has your backout commit at the top.

For what it’s worth, I have more faith in Git for merging stuff like
this smoothly. But today I learnt that the newer Mercurial version
does behave a bit better than it used to.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Terry Reedy]

On 9/28/2015 2:56 AM, Martin Panter wrote:


Also @Terry, not sure on your workflow,


Normal one specified in devguide: commit 3.4, merge 3.5, merge 3.6,
except I do it with TortoiseHG Workbench GUI.

>  but you might have been able

to avoid pushing your 3.4 commits. You might have been able to remove
it locally by using “hg strip”


I need to look into that.


or deleting your local repository.


I have 1 clone for default/3.6 and 3 shares with working directory only 
for 3.5, 3.4, and 2.7.  I believe that deleting 3.4 working directory 
would have no effect as it would be regenerated from default clone.



Or just leave it locally until the other branches got sorted out.


Then I risked having to do a head merge in 2.7 or head merge + forward 
merges in 3.4,5,6 is someone else pushed before I got back to this, 
which I know was going to be several hours, after returning from rushing 
off to work on another project.



Notice that the 3.4 branch still has your backout commit at the top.


Your merges merged my commit and revert forward.  I could have left just 
the commit, but they whoever would have had to decide what to do with 
it.  I though it easier for me to just start over, which I will try now.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Barry Warsaw]

On Sep 28, 2015, at 02:38 PM, Guido van Rossum wrote:

>Your comments make total sense -- we're just short on people who can write
>that kind of docs. :-(

Oh well, maybe someday we will! ;)

-Barry


pgpGVDxk58hN8.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Terry Reedy]

On 9/28/2015 1:12 PM, Alexander Belopolsky wrote:


On Mon, Sep 28, 2015 at 4:13 AM, Terry Reedy > wrote:

Normal one specified in devguide: commit 3.4, merge 3.5, merge 3.6,


That's exactly what I did at fist, but apparently while I was doing
this, another commit was pushed to all three branches.


You lost a push race.  To minimize to possibility of that, I get 
everything ready so that I can commit and merge with under 15 (or 20) 
mouse clicks, without editing anything, after a final pull that does not 
pull anything.  (Sunday, I left out the 'clean repository' check and the 
the final 'nothing found' pull.  I have a .bat file to pull into default 
and update all 4 working directory branches.)  With everything set up, I 
can commit to 2.7 and 3.4 and merge into 3.5 and 3.6 in under a minute. 
 This means having no merge conflicts or having them 'pre-resolved' -- 
see below.



To recover, I did a series of hg update 3.x; hg merge dances

> fighting with Misc/NEWS on every turn.

Hg almost never merges Idle news items correctly, even when there is no 
real conflict, and even when it says there was no conflict.  In the 
latter case, it may have merged in the wrong place, such as putting a 
things into the 3.5.1 section of 3.6 NEWS, which should not be there. 
So I assume that NEWS merges will always be wrong.  After what I hope 
will be the next to last pull, when everything else is done, I open 2.7, 
3.4, 3.5, and 3.6 NEWS on separate tabs (of Notepad++), edit 2.7 and 
save, copy-paste to 3.4 news and save, then copy-paste into 3.5 and 3.6 
WITHOUT saving.  After merging into 3.5, before commiting, I revert NEWS 
to local, switch to Notepad++, being sure to click [No] File changed, 
reload?, save the correct 3.5 NEWS, mark resolved, and commit.  Ditto 
for 3.6.  (I do this in batches, when no one else seems active.)


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Martin Panter]

On 28 September 2015 at 22:31, Terry Reedy  wrote:
> . . . it may have merged in the wrong place, such as putting a things into the
> 3.5.1 section of 3.6 NEWS, which should not be there.

Since you mentioned this and I also noticed this, can I ask why the
3.5.1 section is there, if it isn’t meant to be? There are also other
sections, going back to 3.4.0. They have been there for a while. Is it
just that nobody removed it when it was inherited from 3.5 and 3.4?
When I have merged bug fixes into the default 3.6 branch I have
sometimes let the NEWS entries be automatically be merged into the
3.5.1 section, without updating the 3.6.0 section, after I noticed
other people doing this.

Perhaps we should get rid of the old pre-3.6 sections (perhaps after
adding any missing entries to the 3.6 section)? Otherwise I imagine
some release manager might come along and chop them all out without
adding anything to the 3.6 section.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Eric V. Smith]

At the top of the file in the default branch, I see:

What's New in Python 3.6.0 alpha 1?

Which has a few things in it.

Eric.

On 9/28/2015 7:30 PM, Guido van Rossum wrote:
> It used to be tradition to keep old news sections. Looks like the last
> time we cleared it out was for 3.4.0.
> 
> But it's suspicious there's no section for 3.6 at all. Unless... 3.6 has
> nothing new yet, it's all just bugfixes ported from 3.5?
> 
> On Mon, Sep 28, 2015 at 4:05 PM, Martin Panter  > wrote:
> 
> On 28 September 2015 at 22:31, Terry Reedy  > wrote:
> > . . . it may have merged in the wrong place, such as putting a
> things into the
> > 3.5.1 section of 3.6 NEWS, which should not be there.
> 
> Since you mentioned this and I also noticed this, can I ask why the
> 3.5.1 section is there, if it isn’t meant to be? There are also other
> sections, going back to 3.4.0. They have been there for a while. Is it
> just that nobody removed it when it was inherited from 3.5 and 3.4?
> When I have merged bug fixes into the default 3.6 branch I have
> sometimes let the NEWS entries be automatically be merged into the
> 3.5.1 section, without updating the 3.6.0 section, after I noticed
> other people doing this.
> 
> Perhaps we should get rid of the old pre-3.6 sections (perhaps after
> adding any missing entries to the 3.6 section)? Otherwise I imagine
> some release manager might come along and chop them all out without
> adding anything to the 3.6 section.
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> 
> 
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido )
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
> 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Guido van Rossum]

It used to be tradition to keep old news sections. Looks like the last time
we cleared it out was for 3.4.0.

But it's suspicious there's no section for 3.6 at all. Unless... 3.6 has
nothing new yet, it's all just bugfixes ported from 3.5?

On Mon, Sep 28, 2015 at 4:05 PM, Martin Panter  wrote:

> On 28 September 2015 at 22:31, Terry Reedy  wrote:
> > . . . it may have merged in the wrong place, such as putting a things
> into the
> > 3.5.1 section of 3.6 NEWS, which should not be there.
>
> Since you mentioned this and I also noticed this, can I ask why the
> 3.5.1 section is there, if it isn’t meant to be? There are also other
> sections, going back to 3.4.0. They have been there for a while. Is it
> just that nobody removed it when it was inherited from 3.5 and 3.4?
> When I have merged bug fixes into the default 3.6 branch I have
> sometimes let the NEWS entries be automatically be merged into the
> 3.5.1 section, without updating the 3.6.0 section, after I noticed
> other people doing this.
>
> Perhaps we should get rid of the old pre-3.6 sections (perhaps after
> adding any missing entries to the 3.6 section)? Otherwise I imagine
> some release manager might come along and chop them all out without
> adding anything to the 3.6 section.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Mark Lawrence]

On 29/09/2015 00:52, Nathaniel Smith wrote:

On Mon, Sep 28, 2015 at 2:55 PM, Paul Moore  wrote:

On 28 September 2015 at 22:18, Nathaniel Smith  wrote:

On Sep 28, 2015 1:39 PM, "Paul Moore"  wrote:


On 28 September 2015 at 21:18, MRAB  wrote:

Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
that wasn't due to a bug in the source code.


Interesting. With Python 3.5, what CRT does the module use? With
Python 3.4 and earlier, distutils arranged for mingw builds to link to
the CRT that matched the CPython build,


This doesn't sound right to me. Maybe if you're talking about the original
mingw, but that's been abandoned upstream for years. Nowadays everyone uses
the mingw-w64 fork, for both 32 and 64 bits. And in mingw-w64, the only way
to select a non-default CRT is to hack up the toolchain's source tree and
cross your fingers. It's definitely not a supported configuration upstream.
(Source: we've been talking to mingw-w64 upstream and MS about how to
potentially make building free software less painful on Windows.)


Interesting, it's been a long time since I've done work on the mingw
support, so I hadn't realised this had happened.

I guess that means that nowadays all mingw-compiled extensions link to
msvcrt (I assume that's still the CRT that mingw-w64 uses by default)?


Right, linking to msvcrt.dll is the only standard/supported
configuration for upstream mingw-w64.


but I don't think that
happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
have a CRT that doesn't match the core. Obviously this isn't causing
you issues, but I'm not sure if it could (CRT problems seem to be
mostly about "might cause issues" type problems...)


CRT issues are indeed tricky, because they only bite in certain
circumstances -- so long as you never pass a FILE* or a fileno across the
dll boundary, or call malloc in one module and free in the other, or ...
then it works fine. Except when it doesn't :-).

Example of a subtle issue: msvc and mingw-w64 disagree about whether the x87
fpu should be configured in extended precision mode or not. So simply
loading a mingw-w64-compiled dll can potentially cause math functions in
other dlls to start returning incorrect results. Or maybe not -- it depends
on the details of how MS implemented things, and we can't exactly check. Or
whether you get correct math result in mingw-w64 compiled code can depend on
which runtime was used to spawn the thread that's running the code (!!). And
even if things work today, it could change without warning in any update,
because no one is testing this configuration upstream.

It's totally plausible that the regex module happens to avoid all these
issues. But you can't assume that it's a general solution.


Agreed. That was essentially what I was asking - has someone found a
solution to the issue of avoiding mixed CRTs when using mingw as a
compiler (or at least, found a better solution than using the original
mingw -lmsvcr100 approach combined with the distutils code that
supports this). Looks like the answer is that nobody has. But it's
certainly (still) true that for straightforward cases things seem
pretty much OK, especially if you avoid the "obvious" issues. (The
non-obvious ones sound terrifying, and I'm glad I don't have to write
code that encounters those!!!)


It's not *quite* "nobody has". Carl Kleffner has been doing a lot of
work on trying to set up Python-compatible gcc-based toolchains for
Windows. (This is still pretty much the only hope for getting a usable
free fortran compiler on Windows, and there's still lots of fortran
down at the bottom of the scientific/numerical stack). It's still
rather beta and the approach probably isn't sustainable without more
funding, but right now it's probably the closest thing you can get to
what you're asking about. I believe these are the latest builds:

https://binstar.org/carlkl/mingwpy/files

(Yes, those are .whl files that contain compiler toolchains.)

(Also, if making this *actually* work reliably would make a difference
to anyone, then feel free to send me heartfelt testimonials and I'll
forward them on to MS. Or I guess you could just send them to Steve
Dower directly ;-).)

-n



For anyone interested although this http://bugs.python.org/issue3871 
mingw issue is closed, it points to four other issues.  In total there 
are around 25 mingw issues open.  Maybe some of the work done on them 
can be tied up with the work that Carl Kleffner has done?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 

Re: [Python-Dev] VS 2010 compiler

[Nathaniel Smith]

On Mon, Sep 28, 2015 at 2:55 PM, Paul Moore  wrote:
> On 28 September 2015 at 22:18, Nathaniel Smith  wrote:
>> On Sep 28, 2015 1:39 PM, "Paul Moore"  wrote:
>>>
>>> On 28 September 2015 at 21:18, MRAB  wrote:
>>> > Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
>>> > both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
>>> > that wasn't due to a bug in the source code.
>>>
>>> Interesting. With Python 3.5, what CRT does the module use? With
>>> Python 3.4 and earlier, distutils arranged for mingw builds to link to
>>> the CRT that matched the CPython build,
>>
>> This doesn't sound right to me. Maybe if you're talking about the original
>> mingw, but that's been abandoned upstream for years. Nowadays everyone uses
>> the mingw-w64 fork, for both 32 and 64 bits. And in mingw-w64, the only way
>> to select a non-default CRT is to hack up the toolchain's source tree and
>> cross your fingers. It's definitely not a supported configuration upstream.
>> (Source: we've been talking to mingw-w64 upstream and MS about how to
>> potentially make building free software less painful on Windows.)
>
> Interesting, it's been a long time since I've done work on the mingw
> support, so I hadn't realised this had happened.
>
> I guess that means that nowadays all mingw-compiled extensions link to
> msvcrt (I assume that's still the CRT that mingw-w64 uses by default)?

Right, linking to msvcrt.dll is the only standard/supported
configuration for upstream mingw-w64.

>>> but I don't think that
>>> happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
>>> have a CRT that doesn't match the core. Obviously this isn't causing
>>> you issues, but I'm not sure if it could (CRT problems seem to be
>>> mostly about "might cause issues" type problems...)
>>
>> CRT issues are indeed tricky, because they only bite in certain
>> circumstances -- so long as you never pass a FILE* or a fileno across the
>> dll boundary, or call malloc in one module and free in the other, or ...
>> then it works fine. Except when it doesn't :-).
>>
>> Example of a subtle issue: msvc and mingw-w64 disagree about whether the x87
>> fpu should be configured in extended precision mode or not. So simply
>> loading a mingw-w64-compiled dll can potentially cause math functions in
>> other dlls to start returning incorrect results. Or maybe not -- it depends
>> on the details of how MS implemented things, and we can't exactly check. Or
>> whether you get correct math result in mingw-w64 compiled code can depend on
>> which runtime was used to spawn the thread that's running the code (!!). And
>> even if things work today, it could change without warning in any update,
>> because no one is testing this configuration upstream.
>>
>> It's totally plausible that the regex module happens to avoid all these
>> issues. But you can't assume that it's a general solution.
>
> Agreed. That was essentially what I was asking - has someone found a
> solution to the issue of avoiding mixed CRTs when using mingw as a
> compiler (or at least, found a better solution than using the original
> mingw -lmsvcr100 approach combined with the distutils code that
> supports this). Looks like the answer is that nobody has. But it's
> certainly (still) true that for straightforward cases things seem
> pretty much OK, especially if you avoid the "obvious" issues. (The
> non-obvious ones sound terrifying, and I'm glad I don't have to write
> code that encounters those!!!)

It's not *quite* "nobody has". Carl Kleffner has been doing a lot of
work on trying to set up Python-compatible gcc-based toolchains for
Windows. (This is still pretty much the only hope for getting a usable
free fortran compiler on Windows, and there's still lots of fortran
down at the bottom of the scientific/numerical stack). It's still
rather beta and the approach probably isn't sustainable without more
funding, but right now it's probably the closest thing you can get to
what you're asking about. I believe these are the latest builds:

https://binstar.org/carlkl/mingwpy/files

(Yes, those are .whl files that contain compiler toolchains.)

(Also, if making this *actually* work reliably would make a difference
to anyone, then feel free to send me heartfelt testimonials and I'll
forward them on to MS. Or I guess you could just send them to Steve
Dower directly ;-).)

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Washington DC Python sprint report

[Barry Warsaw]

On Sep 28, 2015, at 08:22 AM, Guido van Rossum wrote:

>I saw that you had a need for an asyncio tutorial. I wonder if the "500
>lines" chapter on asyncio would help? I didn't write it; I only write the
>500 lines of code, A. Jesse Jiryu Davis wrote the text, and it's wonderful:
>http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html

It's a great article, and I particularly like how it starts from first
principles and builds essentially the asyncio framework up from them, with
nice explorations on how things like generators work.

But it's not quite what I'm looking for.  When I go to the stdlib asyncio docs
page, I'm immediately thrown into (albeit good) reference documentation.  I
though that the page "Develop with asyncio" might give me what I want, but
it's really not.

What I'm looking for is a gentle introduction, assuming that the reader either
understands the basics of asynchronous i/o or maybe comes from a background in
threaded programming.  It would then explain things like "this is a Future and
this is what you use it for", "this is a Task and this is when you use a Task
and when you use a Future", "this is when you want to define a Protocol, and
this is how it hooks into the framework".  I think that kind of high-level
overview of the classes and how they hang together is what I think is missing.

Take for example 18.5.3 Tasks and coroutines.  Aside from the fact that the
title would imply it would start explaining Tasks, it actually doesn't get to
them until way later on the page!

What's a Future?  The first thing the asyncio page says is that the "class is
almost compatible with concurrent.futures.Future".  Okay, but that still
doesn't tell me what it *is* or what it's used for.  Following the link to
that other class isn't really enlightening. ;)

I hope my comments make sense.  I know that when you already understand the
concepts, it's difficult to distill them into the right documentation for very
different audiences.  What's there seems mostly great for reference, and maybe
the stdlib docs aren't the right place for what I want, but as the 500 Lines
chapter promotes, asyncio is pretty powerful so it would be nice to have
Python documentation for newcomers.

I'm sure I'm not the right person to write it though. ;)

Cheers,
-Barry


pgpGMWl1WCPhM.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Committing a bug fix

[Terry Reedy]

On 9/28/2015 7:05 PM, Martin Panter wrote:

On 28 September 2015 at 22:31, Terry Reedy  wrote:

. . . it may have merged in the wrong place, such as putting a things into the
3.5.1 section of 3.6 NEWS, which should not be there.


Since you mentioned this and I also noticed this, can I ask why the
3.5.1 section is there, if it isn’t meant to be?


I believe it should not be there and that everything in there should be 
moved to the 3.6.0a1 section (if not already there).  (I already did 
that for IDLE.) Every commit* to a particular branch should be in the 
NEWS section for the next release of that branch.  The purpose of the 
M.N.Pa section is to record commits to the M.N branch that will appear 
in the M.N.Pa release. AFAIK, this has always been the policy until this 
bogus 3.5.1 section was added to 3.6 NEWS.


*Multiple commits for what is conceptually the one patch for one issue 
may get folded together into one NEWS entry, but that does not change 
the gist of the claim.


In 3.5 NEWS, the 3.5.0a1 section follows 3.4.0 (final).  There are no 
3.4.1 etc sections.  All the commits to 3.5 that appeared in 3.5.0a1, 
which branched off from 3.4 at the final release or a bit sooner, are in 
the 3.5.0a1 section.


> There are also other sections, going back to 3.4.0.

There are from before the 3.6 branch was separated from 3.5 (or vice 
versa, if you prefer).  In other words, before there was a 3.6.0a1 to 
commit to.  Whether NEWS should be cumulative or not is a different issue.



When I have merged bug fixes into the default 3.6 branch I have
sometimes let the NEWS entries be automatically be merged into the
3.5.1 section, without updating the 3.6.0 section, after I noticed
other people doing this.


If I am correct as to the intent of each section, this is a mistake. I 
always revert NEWS merges and then save the separately edited file for 
the branch, because hg routinely either creates a (incorrect) conflict 
(for Idle items)* or now, with the incorrect 3.5.1 section in 3.6 NEWS, 
puts things in that incorrect place.


*My guess is that this results from hg merge considering both context 
match and offset when picking the 'best' merge.  For items after the 
Library section, the offset of a correct merge can be 100s of lines.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[Nathaniel Smith]

On Mon, Sep 28, 2015 at 5:14 PM, Mark Lawrence  wrote:
>
> For anyone interested although this http://bugs.python.org/issue3871 mingw
> issue is closed, it points to four other issues.  In total there are around
> 25 mingw issues open.  Maybe some of the work done on them can be tied up
> with the work that Carl Kleffner has done?

There are two different topics here: building extensions with
mingw-w64, and building CPython itself with mingw-w64. That issue is
about the latter, and I think everyone is agreed at this point that
Python.org isn't going to support that so long as there are
incompatible CRT/ABIs between mingw-w64 and MSVC, because that creates
all kinds of problems (e.g. now you have to distinguish between wheels
targeting win32-using-msvc versus win32-using-mingw-w64). So the first
thing to solve is to get the mingw-w64 CRT/ABI incompatibilities
sorted out and extensions working, and then once that's solid we could
think about whether CPython upstream is interested in carrying
mingw-w64 build patches for CPython itself.

(It might not even be that interesting at that point... the main
reason for wanting mingw-w64-built CPython is so that it can
interoperate with the mingw-w64 ecosystem -- e.g. supporting Python
plugins in mingw-w64's gdb. If MSVC and mingw-w64 python become
compatible with each other, then this becomes much less urgent...)

The msys2 folks who are maintaining the CPython-on-mingw-w64 patch set
are definitely aware of Carl's work.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com