Re: [Python-Dev] PEP 0404 and VS 2010

[Kristján Valur Jónsson]

> -Original Message-
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames@python.org] On Behalf Of Christian Tismer
> Sent: 20. nóvember 2013 23:37
> To: Barry Warsaw; python-dev@python.org
> Subject: Re: [Python-Dev] PEP 0404 and VS 2010
> 
> Hey Barry,
> 
> In any case, my question still stands, and I will do something with the
> Stackless branch by end of November. Please influence me ASAP, I don't
> intend to do any harm, but that problem is caused simply by my existence,
> and I want to stick with that for another few decades.
> 
> If I think something must be done, then I have my way to do it.
> If you have good reasons to prevent this, then you should speak up in the
> next 10 days, or it will happen. Is that ok with you?
> 
> Hugs -- your friend Chris


I'd like to add here that at PyCon 2011 (if memory serves me right) I got a 
verbal
agreement from many of you that there would be no objection to me creating
an _unofficial_ "2.8" fork of python.  It could even be hosted on hg.python.org.
I forget if we decided on a name for it, but I remember advocating it as 
"Python classic."

For reasons of work and others, I never got round to creating that branch but
recently Stackless development has picked up the pace to the point where we
feel it necessary to break with strict 2.7 conformance.

Cheers,

Kristján

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] custom thread scheduler to test user synchronisation code

[Dima Tisnek]

Hi,

First, in case such project already exists, could someone point me towards this?

It occurs to me that in some cases may be possible to run exhaustive
test on user-implemented synchronisation code.

let's say, in a simple case, we've got a 2 threads, some user code
using threading.* primitives to synchronise these threads and a unit
test that ensures the logical result of synchronisation is sane.

now, cpython relies on OS to schedule these 2 threads, however, if
this scheduler was under my control, I could force this test case into
many different, distinct thread interleaves. Granted, the total number
of interleaves is exponential and even unbounded in presence of loops.
Yet for small user code and unit test, this could even be exhaustive.

In a brute-force case, a new interleave should be branched off after
every bytecode evaluation (and in case of extensions GIL
acquisition/release). A smarter tester could possibly track primitive
and data accesses by threads and only create interleave branch when
there's potential of something being used by both threads.

How would I go about doing this?

That is how would I go about implementing own thread scheduler?

Thanks,
d.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[martin]

Quoting Barry Warsaw :


On Nov 20, 2013, at 09:52 PM, Christian Tismer wrote:

Many customers are forced to stick with Python 2.X because of other  
products,

but they require a Python 2.X version which can be compiled using Visual
Studio 2010 or better.  This is considered an improvement and not a bug fix,
where I disagree.


I'm not so sure about that.  Python 2.7 can still get patches to help extend
its useful life by allowing it to be built with newer compiler suites.  I
believe this has already been done for various Linux compilers.  I see no
non-technical reason why Python 2.7 can't be taught how to build with VS 2010
or newer.


Unfortunately, there are severe *technical* reasons that make this a  
challenge.

If they wouldn't exist, we would have done this long ago.

The problem is in the ABI, and the question which msvcrt python27.dll links
with. With official support for VS 2010, there would be two ABI-incompatible
releases of Python on Windows; entirely different from supporting a new
version of gcc (say), as newer gccs will typically be binary-compatible
with older releases (I admit I don't understand the ABI compatibility on OSX).

So adding VS 2010 build files to the 2.7 tree would be fine with me, but I
doubt users are helped with that - they want a binary release that uses them.
Releasing this as "Python 2.7", with a "python27.dll" would lead right to
DLL hell.

I could imagine having a separate release that is called "Python 2.7 for
Visual Studio 2010" (and then another one called "Python 2.7 for Visual
Studio 2013"). They could give different names to the Python DLL, e.g.
py27cr100.dll and py27cr110.dll.

Whether this would be a good idea or not, I don't know. It would create
separate ecosystems for different releases of Python 2.7 for different
CRTs. Package authors would have to create multiple binary releases of
the same modules for Windows, and upload them to PyPI. pip would have
to learn to download the right one, depending on what build of Python
2.7 is running.

I'm skeptical that using "Python 2.8" as the release name would help
in the long run. Assuming this is done for VS 2010, then "Python 2.9"
will have to be released for binary compatibility with VS 2013,
and "Python 2.10" for VS 2016, assuming Python 2.7 is then still
around by that time.

Regards,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[martin]

Quoting Nick Coghlan :


Another alternative I'd prefer to an ABI version bump: backporting the "C
runtime independence" aspects of the stable ABI to Python 2.7.


That sounds doable. If we provided a "python2.dll", would could make the
header files using the "restricted API" by default if Python is compiled
with VS 2010. Extension builders could then regularly compile their
extensions with VS 2010, or VS 2013, despite Python itself being linked
with the VS 2008 CRT.

Anybody releasing such binaries would either have to target Python 2.7.7,
or distribute "python2.dll" along with the binary. It should actually
be possible to provide a "python27vs2010.msi" upgrade package that
only deploys the python2.dll and the updated header files, on top of
an existing Python 2.7 installation.

I had originally planned to support the "stable ABI" in Python 2, but
the PEP lagged, and I couldn't finish it in time for the 2.7 release.

If Chris could contribute to make this happen, it would be much
appreciated.

Regards,
Martin

P.S. Thinking about this, there are some issues. The "restricted API"
hides the object layout of all objects, in particular of type objects.
Adding the PEP 384 API (PyType_FromSpec) might be a bit heavy for 2.7.

So it might by better to provide a "py27compat.dll" instead which does
not hide the structures (as they won't change during the remaining life
of 2.7), but only hides any APIs and macros that directly expose CRT
functionality.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Paul Moore]

On 21 November 2013 11:15,   wrote:
> Whether this would be a good idea or not, I don't know. It would create
> separate ecosystems for different releases of Python 2.7 for different
> CRTs. Package authors would have to create multiple binary releases of
> the same modules for Windows, and upload them to PyPI. pip would have
> to learn to download the right one, depending on what build of Python
> 2.7 is running.

It would also mean that the wheel compatibility tags for Windows would
no longer work, as currently the code makes no provision for multiple
ABIs on Windows within a single Python minor version. So it would not
be possible to reliably release binary wheels which could be verified
as compatible with the Python version you're installing them to (a key
benefit of wheels over earlier binary formats, which did not consider
ABI compatibility at all).

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Antoine Pitrou]

On Thu, 21 Nov 2013 09:19:27 +
Kristján Valur Jónsson  wrote:
> 
> For reasons of work and others, I never got round to creating that branch but
> recently Stackless development has picked up the pace to the point where we
> feel it necessary to break with strict 2.7 conformance.

Why is that? Stackless can't switch to 3.x?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Kristján Valur Jónsson]

> -Original Message-
> From: Python-Dev [mailto:python-dev-
> bounces+kristjan=ccpgames@python.org] On Behalf Of Antoine Pitrou
> Sent: 21. nóvember 2013 12:06
> To: python-dev@python.org
> Subject: Re: [Python-Dev] PEP 0404 and VS 2010
> 
> On Thu, 21 Nov 2013 09:19:27 +
> Kristján Valur Jónsson  wrote:
> >
> > For reasons of work and others, I never got round to creating that
> > branch but recently Stackless development has picked up the pace to
> > the point where we feel it necessary to break with strict 2.7 conformance.
> 
> Why is that? Stackless can't switch to 3.x?
> 
Yes, we have stackless 3.3
But there is desire to have a 2.X version, with added fixes from 3.x, e.g. 
certain improvements in the
standard library etc.
It's the old argument:  moving to 3.x is not an option for some users, but 
there are known improvements that
can be applied to current 2.7.  Why not both have our cake and eat it?
cPython had probably two driving concerns for not making a 2.8:
1) Focussing development on one branch
2) encouraging (forcing) users to take the leap to 3 come hell or high water.

For Stackless, neither argument applies because 2.8 work would be done by us 
and stackless has no particular allegiance towards either version.

K
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Victor Stinner]

2013/11/21 Charles-François Natali :
> I'm happy to officially accept PEP 454 aka tracemalloc.
> The API has substantially improved over the past weeks, and is now
> both easy to use and suitable as a fundation for high-level tools for
> memory-profiling.
>
> Thanks to Victor for his work!

Thanks Charles-François for your help!

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[martin]

Quoting Greg Ewing :


Concerning the version number, I thought the intention of
PEP 404 was simply to say that the PSF would not be releasing
anything called Python 2.8, not to forbid anyone *else*
from doing so.

Or am I wrong about that?


That's correct.


If I'm right, there's nothing stopping Christian from
releasing Stackless Python 2.8 with whatever improvements
he wants.


"Nothing stopping" is exactly right. People still don't need
to like it. Barry wishes there was something stopping him,
and be it a lawyer invoking trademark law.

If "2.8" was just a version number of Stackless Python not
related to Python version (like PyPy's version number currently
being 2.2), protest would be much less. People fear that
releasing Stackless Python 2.8 would create the impression that
even CPython has continued development, and it might require
significant PR efforts to educate people.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Paul Moore]

On 21 November 2013 21:02, Greg Ewing  wrote:
> Is that much different from package authors having to
> release binaries for different versions of Python,
> if they want to support older versions?
>
> Having multiple binaries for the same x.y version
> is different from what's been done before, but it
> seems to me an unavoidable consequence of supporting
> one x.y version for longer than usual.

None of the currently available binary distribution formats
distinguish Windows binaries by anything other than minor version. For
wheels (and I think eggs), this is a showstopper as the name is
essential metadata (compatibility tags) for the other formats (wininst
and msi) the name is merely informational - packagers could rename,
but (a) they will forget, and (b) the users won't know if they have or
not.

Before we can cleanly support multiple ABIs for a single minor version
on Windows, we need to have a resolution of this dilemma (which may be
nothing more than "only binaries for the python.org builds are allowed
on PyPI"...)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Antoine Pitrou]

On Thu, 21 Nov 2013 12:36:48 +
Kristján Valur Jónsson  wrote:
> Yes, we have stackless 3.3
> But there is desire to have a 2.X version, with added fixes from 3.x, e.g. 
> certain improvements in the
> standard library etc.
> It's the old argument:  moving to 3.x is not an option for some users, but 
> there are known improvements that
> can be applied to current 2.7.  Why not both have our cake and eat it?
> cPython had probably two driving concerns for not making a 2.8:
> 1) Focussing development on one branch
> 2) encouraging (forcing) users to take the leap to 3 come hell or high water.
> 
> For Stackless, neither argument applies because 2.8 work would be done
> by us and stackless has no particular allegiance towards either version.

Stackless can release their own Stackless 2.8 if they want, but I don't
get why CPython would have a 2.8 too.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Nick Coghlan]

On 22 Nov 2013 07:43, "Charles-François Natali"  wrote:
>
> Hi,
>
> I'm happy to officially accept PEP 454 aka tracemalloc.
> The API has substantially improved over the past weeks, and is now
> both easy to use and suitable as a fundation for high-level tools for
> memory-profiling.
>
> Thanks to Victor for his work!

Huzzah! Thanks to you both for getting this ready for inclusion :)

Cheers,
Nick.

>
>
> Charles-François
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Daniel Holth]

On Thu, Nov 21, 2013 at 4:12 PM, Paul Moore  wrote:
> On 21 November 2013 21:02, Greg Ewing  wrote:
>> Is that much different from package authors having to
>> release binaries for different versions of Python,
>> if they want to support older versions?
>>
>> Having multiple binaries for the same x.y version
>> is different from what's been done before, but it
>> seems to me an unavoidable consequence of supporting
>> one x.y version for longer than usual.
>
> None of the currently available binary distribution formats
> distinguish Windows binaries by anything other than minor version. For
> wheels (and I think eggs), this is a showstopper as the name is
> essential metadata (compatibility tags) for the other formats (wininst
> and msi) the name is merely informational - packagers could rename,
> but (a) they will forget, and (b) the users won't know if they have or
> not.
>
> Before we can cleanly support multiple ABIs for a single minor version
> on Windows, we need to have a resolution of this dilemma (which may be
> nothing more than "only binaries for the python.org builds are allowed
> on PyPI"...)
>
> Paul

As for wheel it actually does support an ABI tag that is separate from
the Python version and the architecture. It's the second one
pyversion-abi-architecture as in py27-none-any or py27-cp27-linux_i386
(spelling?). The build tool and installer would have to be modified to
be aware of any newly defined ABI tags.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 1:32 PM, Christian Tismer wrote:

>  I am converted to an OS X developer since 2006, but never had ABI
> problems,
> because I use homebrew,
>

Right, different story -- you are supposed to compile everything on the
target system, so everything stays compatible.

but instead of being set free on Windows after 30 years
> of pain, I now have the same mess in my Parallels VMs.
> Customers are so cruel, aren't they?
>

business would be som much easier without them [but less successful ;-) ]

Do you want people to be able to use extensions built by third parties for
> python.org CPython with your binary builds?
>
> Would be great, but I would not mind to create their extensions on
> stackless.com, instead.
>
The other potential route for error is a pip install -- you don't want pip
> to find a binary that is incompatible with your build -- so you should
> assure that whatever pip/wheel uses to identify the build is
> set differently in your build (see the relevant PEPs).
>
>
> Yes, I want to make PIP work with it, want to make it very simple to
> install
> whatnot, and let people use that stuff. So if you can, please teach me
> what I need to do or avoid.
>

I think it's a "simple" as making sure that the names your binary generate
for pip and wheel is different than the python.org VS2008 build. That way,
if someone does "pip install something" with your binary, it won't
accidentally download a binary wheel, but rather will try to install from
source. And if their complier is set up right, that should work (ignore
dependency hell for now).

And you could build wheels and host them on the stackless site for things
that aren't easy to build.

folks could still get in trouble trying to install msi installers built for
pyton,org python -- but it sounds like you could fix that with a dll name
change.

What I want is a workable CPython path for some customer (!=CCP) to use
> for the next (maybe 5) years, and I want to build that now, for good.
>

That's the trick -- I don't think it's only me that thinks we should have
the option, some point in the future, to put out VS_something_else binary
of Python2.7, or a future version.

So it would be nice to come to a consensus on the pip/wheel/dll naming
scheme before you put anything out.

 On Thu, Nov 21, 2013 at 1:31 PM, Daniel Holth  wrote:

> As for wheel it actually does support an ABI tag that is separate from
> the Python version and the architecture. It's the second one
> pyversion-abi-architecture as in py27-none-any or py27-cp27-linux_i386
> (spelling?). The build tool and installer would have to be modified to
> be aware of any newly defined ABI tags.


I'm still confused a bit about ABI vs. Platform vs. I had thought that ABI
meant the ABI defined by the python source code, rather than the
ABI defined by the complier -- the examples in the doc imply that.

As for "platform", I'm not sure how to define that, but it seems reasonable
to consider the run-time core Clib as part of the "platform". In any case,
that's what the Mac builds are doing.

But it really doesn't matter which you use, as long as there is
some consistency.

Maybe this part is a topic for the distutils-sig.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Heimes]

Am 21.11.2013 16:12, schrieb Barry Warsaw:
> On Nov 21, 2013, at 02:16 PM, Kristján Valur Jónsson wrote:
> 
>> Oh, this is the misunderstanding.  No one is trying to get permission for
>> "CPython 2.8", only "Stackless Python 2.8".
> 
> I think this is a very bad idea.  We've worked hard to send the message that
> the migration path is to Python 3 and while I understand that many people
> don't want to or cannot switch to Python 3, backpeddling on this message from
> the Python community will be bad PR and open the floodgates for pressure to
> continue the Python 2 lineage.
> 
> Stackless can of course do whatever it wants, but I would strongly oppose the
> use of the word "Python" in that.  I don't want to be antagonistic, but I
> think the PSF should also oppose allowing the use of the trademark for
> any association with an unofficial 2.8.

Yes, please don't use a name that contains both the strings "Python" and
"2.8". It's going to create lots and lots of confusion. I strongly urge
you to call it "Stackless 2.8" or something similar.

Christian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Ethan Furman]

On 11/21/2013 10:53 AM, Christian Tismer wrote:


So even if VS2010 exists only in the stackless branch, it is very likely
to get used as CPython VS 2010, and I again have the naming problem ...


What's wrong with calling it CPython VS 2010? And Stackless VS 2010?

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] flaky tests caused by repr() sort order

[Christian Heimes]

Am 21.11.2013 18:57, schrieb Tim Peters:
> Best to change the failing tests.  For example, _they_ can sort the
> dict keys if they rely on a fixed order.  Sorting in general is a
> dubious idea because it can be a major expense with no real benefit
> for most uses.

I don't consider repr() as a performance critical function. It's mostly
used for debugging.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Nick Coghlan]

On 22 Nov 2013 02:03, "Chris Barker - NOAA Federal" 
wrote:
>
>
>>
>> with older releases (I admit I don't understand the ABI compatibility on
OSX).
>
>
> Well, with OS-X, it's not exactly the C lic in the same way, but there
are different SDKs for different OS versions, and you can add to that PPC
vs Intel processors and 32 vs 64 bit.
>
> So we have for years had two builds for OS-X, and you can add to that
Macports, Homebrew, and what have you.
>
> And the idea that this isn't an issue for gcc makes no sense-- it's such
a big issue for Linux, in fact that python.org doesn't even try to build
binaries for Linux, and pypi has disabled binary wheels for Linux.

Indeed :)

For 2.7.7, I think some combination of the two following ideas would be
worth pursuing:

- a C runtime independent API flag (set by default on Windows when building
with a compiler other than VS2008). This would largely be a backport of
some of the stable ABI work from Python 3.
- getting Windows closer to the current Mac OS X situation by ensuring that
the C runtime used directly affects the ABI flags and shared library names.
PyPI would apply the Mac OS X guideline where extensions are expected to be
compatible with the python.org binaries.

This would be the biggest change pushed through under the "make builds
work" policy for the extended 2.7 lifecycle, but Microsoft's aggressive
approach to deprecating old compilers and C runtimes means I think we don't
have much choice.

In the near term, if Stackless build to a different DLL name under VS2010
and make it clear to their users that extension compatibility issues are
possible (or even likely) if they aren't rebuilt from source, then I think
that would be compatible with the above proposal for a way forward.

Then we'd just need some volunteers to write and implement a PEP or two :)

(Note, similar to the Mac OS X situation, I think we should do this without
hosting any new interpreter variants on python.org - VS2010 and VS2013
source builds would become separate build-from-source ecosystems for
extensions, using sdists on PyPI as the default distribution mechanism)

Cheers,
Nick.



>
>
>>
>> So adding VS 2010 build files to the 2.7 tree would be fine with me,
>
>
> I think this part is a no brainer.
>
> I also think having a 2.8 out there that is exactly the same as 2.7,
except that it was built with a different version of a compiler on one
particular platform is a very very bad idea.
>
> So how CAN this be addressed? This is really a distribution issue, and
the distutils-sig has been hard at work building the infrastructure to
support things like this.
>
> Right now, if you build binary wheels with the two different "official"
OS-X builds, you will get two different names, and pop can find the correct
one. (last I checked, pip would happily install the wrong one if you asked
it to, though I think that is slated to be fixed)
>
> So if a different build of 2.7 for Windows is put out there, we need to
make sure it reports the platform in a way that wheel and pip can make the
distinction.
>
> It might be nice to patch the win_inst too--IIRC it's still not very
smart about even 32 vs 64 bit builds.
>
> As for stackless--just to be clear--can you use extensions built with the
"regular" python with a stack less binary? If so, I understand the concern.
If not, then it seems stackless is a separate ecosystem anyway.
>
> Chris
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Victor Stinner]

2013/11/21 Nick Coghlan :
> Huzzah! Thanks to you both for getting this ready for inclusion :)

I now hope that someone will use it :-)


By the way, collections.namedtuple has a private _source attribute.
This attributes uses something like 676.2 kB in the Python test suite,
it the 5th biggest user of memory.

Woud you mind if I simply remove it?

I'm asking because this *private* attribute is documented, which is
surprising for something private. I don't understand the use case. Is
it to debug the implementation of the namedtuple? Or I imagined a
different usecase: dump the _source into a file to speedup the startup
time. But this optimization is not used in CPython whereas we care of
the startup time.

Can maybe Raymond explain the use case for this attribute?

If there is a real use case, I would prefer a real function to get the
source code of a function defining a new namedtuple type.

I already opened an issue for that:
http://bugs.python.org/issue19640

--

The implementation of namedtuple constructor ("factory" ?) looks
inefficient. Why do we need to build source code to then parse the
code source? Why not having a base namedtuple class and then inherit
from this class? It may reduce the memory footprint, allow to check if
a type is namedtuple, etc.

There are two different concerns, I prefer to only focus on the
removal of the _source attribute. But it's also to say the the _source
attribute is very specific to the current (inefficient?) implement of
the namedtuple constructor.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Paul Moore]

On 21 November 2013 21:27, Chris Barker  wrote:
> That's already the unstated case. But besides stackless, it some of us are
> advocating that there be python.org-provided binaries built with a newer
> compiler (eventually, anyway).

I see no problem with python.org producing and distributing a
VS2010-compiled version of Python 2.7 (subject to python-dev being
willing to do so of course). But in order to do so, it would be
necessary to address the issue of having multiple ABIs within one
minor version on Windows. As you say, that's just a matter of doing
things like enabling the ABI tag in wheel, making the necessary
distutils changes (don't forget that distutils validates the MSVC
version in use) and whatever other things I've forgotten about.

It's by no means impossible, but nor is it trivial to do it "right".

On the other hand, as Monty Python would say, "2.8 is right out" :-)

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Glenn Linderman]

On 11/21/2013 12:23 PM, Christian Tismer wrote:

Maybe I would generate a cpython and spython exe and support them
both in the same distribution? 

That sounds cool, if possible.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 1:09 PM, Greg Ewing wrote:

> Concerning the version number, I thought the intention of
> PEP 404 was simply to say that the PSF would not be releasing
> anything called Python 2.8, not to forbid anyone *else*
> from doing so.
>
> Or am I wrong about that?
>
> If I'm right, there's nothing stopping Christian from
> releasing Stackless Python 2.8 with whatever improvements
> he wants. If it includes other improvements beyond the
> change of compiler, it may even deserve the higher
> version number.


If so, I still think it's a very bad idea to use a version number bump for
a compiler change -- that's not at all what the version number means. That
version number specifies a version of the code, and a version of the
language (and library, not a particular ABI -- after all, how many
different binary versions of python2.7 are out there now? Multiple
platforms, multiple bit-depths, etc.

-Chris








>
> --
> Greg
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> chris.barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Nick Coghlan]

On 22 Nov 2013 09:02, "Victor Stinner"  wrote:
>
> 2013/11/21 Nick Coghlan :
> > Huzzah! Thanks to you both for getting this ready for inclusion :)
>
> I now hope that someone will use it :-)
>
>
> By the way, collections.namedtuple has a private _source attribute.
> This attributes uses something like 676.2 kB in the Python test suite,
> it the 5th biggest user of memory.
>
> Woud you mind if I simply remove it?
>
> I'm asking because this *private* attribute is documented, which is
> surprising for something private. I don't understand the use case. Is
> it to debug the implementation of the namedtuple? Or I imagined a
> different usecase: dump the _source into a file to speedup the startup
> time. But this optimization is not used in CPython whereas we care of
> the startup time.

namedtuple deviates from normal naming conventions to avoid name clashes
between field names and the attributes and methods of the object itself
(this is covered in the docs for namedtuple).

Skipping saving _source under -OO would probably be a good thing, but
otherwise it's a public API with the usual backwards compatibility
guarantees.

Cheers,
Nick.

>
> Can maybe Raymond explain the use case for this attribute?
>
> If there is a real use case, I would prefer a real function to get the
> source code of a function defining a new namedtuple type.
>
> I already opened an issue for that:
> http://bugs.python.org/issue19640
>
> --
>
> The implementation of namedtuple constructor ("factory" ?) looks
> inefficient. Why do we need to build source code to then parse the
> code source? Why not having a base namedtuple class and then inherit
> from this class? It may reduce the memory footprint, allow to check if
> a type is namedtuple, etc.
>
> There are two different concerns, I prefer to only focus on the
> removal of the _source attribute. But it's also to say the the _source
> attribute is very specific to the current (inefficient?) implement of
> the namedtuple constructor.
>
> Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Tismer]

On 21/11/13 19:59, Ethan Furman wrote:

On 11/21/2013 10:53 AM, Christian Tismer wrote:


So even if VS2010 exists only in the stackless branch, it is very likely
to get used as CPython VS 2010, and I again have the naming problem ...


What's wrong with calling it CPython VS 2010? And Stackless VS 2010?


Sounds tempting in the first place, but then I need to change policy a bit.
With the compiler switch, we always produced the identical Python version
where Stackless is based upon. Well, this version string is not a problem.

You mean I would stick with the latest CPython version numbers, produce
maybe different dll names (maybe cpy27vs2010.dll and spy27vs2010.dll)
which could even live together?

Maybe I would generate a cpython and spython exe and support them
both in the same distribution?

What would happen with binary extensions on PyPI? Would they install
and then fail, or is the distinction based upon the name "python"?

If the latter is true, I think I could live with that, and build binary 
packages

for all relevant extensions.

At first sight I was about to complain, at second sight I am about to fall
in the trap of being happy. I do not want to loose our relation to CPython
and do any harm. But if there is such an easy solution, I might think to
go for it.
In the end, I want to go the path that MvL/Nick proposed.
But as a quick solution for now, I would love to have an easy first 
solution.


Please do not take me too seriously, I need to sleep over this to see if the
idea stands my nightmares.

cheers & thanks -- Chris

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] flaky tests caused by repr() sort order

[Tim Peters]

[Christian Heimes]
> the buildbots are flaky because two repr() tests for userdict and
> functools.partial fail every now and then. The test cases depend on a
> fixed order of keyword arguments the representation of userdict and
> partial instances. The improved hash randomization of PEP 456 shows its
> power. I haven't seen the issue before because it happens rarely and
> mostly on 32 bit platforms.
>
> http://bugs.python.org/issue19681
> http://bugs.python.org/issue19664
>
> I'm not sure about the best approach for the issues. Either we need to
> change the test case and make it more resilient or the code for repr()
> must sort its dict keys.

Best to change the failing tests.  For example, _they_ can sort the
dict keys if they rely on a fixed order.  Sorting in general is a
dubious idea because it can be a major expense with no real benefit
for most uses.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Terry Reedy]

On 11/21/2013 5:13 PM, mar...@v.loewis.de wrote:


Quoting Greg Ewing :


Concerning the version number, I thought the intention of
PEP 404 was simply to say that the PSF would not be releasing
anything called Python 2.8, not to forbid anyone *else*
from doing so.

Or am I wrong about that?


That's correct.


If I'm right, there's nothing stopping Christian from
releasing Stackless Python 2.8 with whatever improvements
he wants.


"Nothing stopping" is exactly right. People still don't need
to like it. Barry wishes there was something stopping him,
and be it a lawyer invoking trademark law.


My lay knowledge of US Trademark law and case history and reading of
http://www.python.org/psf/trademarks/
suggests that 'nothing stopping' is exactly wrong. I believe the 
trademark has also been registered in Europe.


As usual, 'I am not a lawyer', but if Christian wants to push forward 
with using 'Python 2.8', I suggest that he consult the PSF Trademark 
Committee and lawyer first.



If "2.8" was just a version number of Stackless Python not
related to Python version (like PyPy's version number currently
being 2.2), protest would be much less.


But it is *not* unrelated.


People fear that
releasing Stackless Python 2.8 would create the impression that
even CPython has continued development, and it might require
significant PR efforts to educate people.


Yep, I do. 'Stackless 10' would have no issue.

---
I think two unrelated issues are being mixed together in this thread 
that really should be two separate thread.


1. Compiling Windows CPython x.y with more than one compiler. This is 
not specifically a Stackless issue or even a 2.7 issue. If 3.4 is 
released compiled with VS2010, there will be people wanting it compiled 
with VS2013(12?). And vice versa.


2. Public releases of new Python versions based on 2.7 that are not 3.x. 
How they are named is an issue regardless of what Windows compiler is 
used, if indeed a release even uses one. In my view, either no one 
should be allowed to call something 'X Python 2.8' (or any unofficial 
x.y), or everyone should. The latter might mean that we see Stackless 
Python 2.8, Ubuntu Python 2.8, RedHat Python 2.8, ActiveState Python 
2.8, Enthought Python 2.8, etc, all different. I prefer no one.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 428 - pathlib - ready for approval

[Brett Cannon]

On Wed, Nov 20, 2013 at 5:41 PM, Mark Lawrence wrote:

> On 20/11/2013 22:01, Antoine Pitrou wrote:
>
>>
>> pathlib imports many modules at startup, so for scripts for which
>> startup time is critical using os.path may still be the best option.
>>
>>
> Will there be or is there a note to this effect in the docs?


Could be a comment in the source but it should not be in the docs. Only
python-dev would care about this and actual users shouldn't over-optimize
and avoid pathlib because it imports stuff.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Barry Warsaw]

On Nov 21, 2013, at 06:36 PM, Terry Reedy wrote:

>As usual, 'I am not a lawyer', but if Christian wants to push forward with
>using 'Python 2.8', I suggest that he consult the PSF Trademark Committee and
>lawyer first.

Just to make clear, I'm definitely *not* suggesting this particular case ever
escalate to such a level.  Chris is our friend and a nice guy. :)

I'm just saying that I think we should avoid any possible confusion on the
part of ours (and Stackless's) users.  And based on the discussions here,
there are plenty of good alternatives.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Antoine Pitrou]

On Thu, 21 Nov 2013 18:43:37 -0500
Barry Warsaw  wrote:

> On Nov 21, 2013, at 06:36 PM, Terry Reedy wrote:
> 
> >As usual, 'I am not a lawyer', but if Christian wants to push forward with
> >using 'Python 2.8', I suggest that he consult the PSF Trademark Committee and
> >lawyer first.
> 
> Just to make clear, I'm definitely *not* suggesting this particular case ever
> escalate to such a level.  Chris is our friend and a nice guy. :)
> 
> I'm just saying that I think we should avoid any possible confusion on the
> part of ours (and Stackless's) users.  And based on the discussions here,
> there are plenty of good alternatives.

+1 from me :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Steve Dower]

Nick Coghlan wrote:
> On 21 Nov 2013 10:33, "Antoine Pitrou"  wrote:
>> I think it isn't only about teaching it to build with VS 2010, but
>> providing binaries compatible with the VS 2010 runtime.
>> Otherwise, AFAIU, if extensions are built with VS 2010 but loader with
>> a VS 2008-compiled Python, there will be ABI problems.
>
> Right, this is the problem - building with newer compilers isn't an issue,
> changing the Windows ABI *is* (which is the reason Christian is proposing a
> version bump to denote the ABI incompatibility).
>
> If Visual Studio Express 2010 and later can be told to build against the 
> VS2008
> C runtime, then that is something that can (and probably should) be enabled in
> the next CPython 2.7 maintenance release (for both the main build process and
> for distutils extension building).

Unfortunately, this is not the case. The compiler has intimate knowledge of the 
associated CRT and can't build against other versions (officially; if there's 
an unofficial way I genuinely don't know what it is). It is possible to use the 
older compiler from newer versions of Visual Studio, but that doesn't really 
solve any issues.

> Doing a 2.8 release *just* to change the Windows ABI to a new version of the 
> MS
> C runtime, on the other hand, seems impossible to do without thoroughly
> confusing people (regardless of whether it's CPython or Stackless making such 
> a
> change).

I agree, this is a bad idea. We'd also need to update distutils to detect the 
right version, and if we can do this then it's easier to update it to detect 
VC9 when installed from 
http://www.microsoft.com/en-us/download/details.aspx?id=3138 - currently it 
only finds VS2008, which installs things slightly differently, despite being an 
identical compiler.

I've had one go at making the change (http://bugs.python.org/issue7511), though 
that (a) had a bug and (b) broke people who monkey patch distutils to return a 
different vcvarsall.bat (including the distutils tests - my fix removed the 
dependency on vcvarsall.bat completely). Happy to revise it if there's interest 
in taking the change.

> I'd certainly want to explore all our alternatives with the Microsoft folks 
> for
> getting our preferred option (new compiler, old C runtime) working in an open
> source friendly way before we went down the path of a 2.x ABI bump.

In my experience, extensions built with later compilers work fine, provided you 
stay away from the FILE* APIs. One of my side projects is C++11 bindings for 
hosting/extending Python, and I run my tests with VC12 (Visual Studio 2013) 
against python.org's 2.6, 2.7, 3.2 and 3.3 binaries without issue.

What may be best is to detect MSVC>9.0 in the headers and simply block the 
dangerous APIs when building sdists. Then extensions can be built with any 
available version of MSVC, provided they are "safe", and the build will fail if 
they aren't.

There are still some ways the differing CRTs can cause issues - alloc/free 
pairing, for example - so some macros may also have to become exported 
functions. After that I don't think there are any ways to accidentally cause 
issues; you'd really have to be doing strange things (such as passing pointers 
to CRT types between extensions via capsules).

> I've cc'ed Steve Dower directly, as he's the best person I know to ask about
> this kind of problem.

Cheers,
Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] flaky tests caused by repr() sort order

[Daniel Holth]

+1 on unsorted repr(). It makes it obvious that the collection is not sorted.

On Thu, Nov 21, 2013 at 1:10 PM, Christian Heimes  wrote:
> Am 21.11.2013 18:57, schrieb Tim Peters:
>> Best to change the failing tests.  For example, _they_ can sort the
>> dict keys if they rely on a fixed order.  Sorting in general is a
>> dubious idea because it can be a major expense with no real benefit
>> for most uses.
>
> I don't consider repr() as a performance critical function. It's mostly
> used for debugging.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Tismer]

On 22.11.13 00:53, Antoine Pitrou wrote:

On Thu, 21 Nov 2013 18:43:37 -0500
Barry Warsaw  wrote:


On Nov 21, 2013, at 06:36 PM, Terry Reedy wrote:


As usual, 'I am not a lawyer', but if Christian wants to push forward with
using 'Python 2.8', I suggest that he consult the PSF Trademark Committee and
lawyer first.

Just to make clear, I'm definitely *not* suggesting this particular case ever
escalate to such a level.  Chris is our friend and a nice guy. :)

I'm just saying that I think we should avoid any possible confusion on the
part of ours (and Stackless's) users.  And based on the discussions here,
there are plenty of good alternatives.

+1 from me :-)


Thank you for that input! It was important and urgent, as I saw myself 
jumping

into the wrong wagon, again.

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Barry Warsaw]

On Nov 21, 2013, at 02:16 PM, Kristján Valur Jónsson wrote:

>Oh, this is the misunderstanding.  No one is trying to get permission for
>"CPython 2.8", only "Stackless Python 2.8".

I think this is a very bad idea.  We've worked hard to send the message that
the migration path is to Python 3 and while I understand that many people
don't want to or cannot switch to Python 3, backpeddling on this message from
the Python community will be bad PR and open the floodgates for pressure to
continue the Python 2 lineage.

Stackless can of course do whatever it wants, but I would strongly oppose the
use of the word "Python" in that.  I don't want to be antagonistic, but I
think the PSF should also oppose allowing the use of the trademark for
any association with an unofficial 2.8.

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Antoine Pitrou]

On Fri, 22 Nov 2013 09:17:14 +1000
Nick Coghlan  wrote:
> 
> Skipping saving _source under -OO would probably be a good thing, but
> otherwise it's a public API with the usual backwards compatibility
> guarantees.

I think skipping saving _source under -OO should be a bugfix. It's
terribly weird and suboptimal to remove docstrings but keep the
source code of namedtuple instances in some obscure attributes.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 454 - tracemalloc - accepted

[Eric Snow]

On Thu, Nov 21, 2013 at 4:17 PM, Nick Coghlan  wrote:
> Skipping saving _source under -OO would probably be a good thing, but
> otherwise it's a public API with the usual backwards compatibility
> guarantees.

One alternative might be to make it a property that re-generates the
source (you just need self.__class__.__name__ and self._fields).

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Nick Coghlan]

On 21 November 2013 22:51, Christian Heimes  wrote:
> Am 21.11.2013 12:31, schrieb mar...@v.loewis.de:
>> That sounds doable. If we provided a "python2.dll", would could make the
>> header files using the "restricted API" by default if Python is compiled
>> with VS 2010. Extension builders could then regularly compile their
>> extensions with VS 2010, or VS 2013, despite Python itself being linked
>> with the VS 2008 CRT.
>
> What about the CAPI functions like PyFile_FromFile() and PyFile_AsFile()
> that take a FILE* as argument? Or functions like PyErr_SetFromErrno()
> that use the errno thread local variable? These function will either
> crash or not work properly with mixed CRTs. Every CRT has its own errno
> TLS, so Python won't see the errno of a CRT100 function like open(2),
> see http://bugs.python.org/issue15883 .
>
> I don't understand how a stable Python ABI solves the issue of unstable
> CRT ABIs. Are you planning to address these issues?

The stable ABI excludes all the CRT dependent APIs - that is exactly
the subset of the stable ABI restrictions that I am suggesting we
should backport to 2.7.7 rather than Stackless making a 2.8 release
just to update to a backwards incompatible C runtime on Windows.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Nick Coghlan]

On 22 November 2013 00:16, Kristján Valur Jónsson  wrote:
>
>> > For Stackless, neither argument applies because 2.8 work would be done
>> > by us and stackless has no particular allegiance towards either version.
>>
>> Stackless can release their own Stackless 2.8 if they want, but I don't get 
>> why
>> CPython would have a 2.8 too.
>
> Oh, this is the misunderstanding.  No one is trying to get permission for 
> "CPython 2.8",
> only "Stackless Python 2.8".
>
> The "namespace" question from Christian has to do with a "python28.dll" which 
> would be
> built using VS2010, that this would never clash with a CPython version built 
> the
> same way.   such clashes would be very unfortunate.
>
> Of course, we could even make a full break, if there will never be a CPython 
> 2.8 (which there won't be)
> and call the dll slpython28.dll.

OK, rereading Christian's original post, I think we may need to
understand the problem you are trying to solve a little better. If I
have understood correctly,

1. Currently CPython 2.7 only officially supports building with Visual
Studio 2008. This is due to the ABI instability of the Windows CRT and
the fact that elements of the CRT ABI are republished as part of the
Python ABI.

2. (CCP?) customers are wanting to build CPython *from source*, and
don't care about compatibility with binary extensions built by others
in the normal way against the regular CPython release. If they need
extensions, they will build them from source themselves, thus avoiding
any CRT ABI incompatibility issues.

If this interpretation is correct, then Martin's suggestion of a
different DLL name entirely *should work* for your use case. Something
like:

slpy27cr100.dll ("Stackless Python 2.7 for Visual Studio 2010")
slpy27cr110.dll ("Stackless Python 2.7 for Visual Studio 2013")

If there's no need for a compatible ecosystem of C extensions (e.g.
it's being embedded as part of a large application which takes care of
linking everything properly), then the problem becomes much simpler,
and there's a much higher chance it can be handled upstream in CPython
itself.

The only thing that *can't* happen is to build a python27.dll that
links against a newer C runtime than the Visual Studio 2008 one.

I believe this approach would be substantially less confusing for the
broader community than trying to explain that a 2.8 release of
Stackless Python was really just a version bump for the Windows CRT
ABI.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Update list of accepted but not yet implemented or merged peps.

[Victor Stinner]

Hum, I don't think that regex module will enter Python 3.4 before this
week-end, there is no PEP.

For the "Introspection information for builtins", I think the PEP 436
has been accepted. The code has been merged, but the PEP status is
still draft.

Victor

2013/11/22 barry.warsaw :
> http://hg.python.org/peps/rev/ea82fe7aa1ff
> changeset:   5311:ea82fe7aa1ff
> user:Barry Warsaw 
> date:Thu Nov 21 18:21:06 2013 -0500
> summary:
>   Update list of accepted but not yet implemented or merged peps.
>
> files:
>   pep-0429.txt |  14 +-
>   1 files changed, 9 insertions(+), 5 deletions(-)
>
>
> diff --git a/pep-0429.txt b/pep-0429.txt
> --- a/pep-0429.txt
> +++ b/pep-0429.txt
> @@ -74,6 +74,15 @@
>  * PEP 446, make newly created file descriptors not inherited by default
>  * PEP 450, basic statistics module for the standard library
>
> +Accepted but not yet implemented/merged:
> +
> +* PEP 428, the pathlib module -- object-oriented filesystem paths
> +* PEP 451, a ModuleSpec Type for the Import System
> +* PEP 453, pip bootstrapping/bundling with CPython
> +* PEP 454, the tracemalloc module for tracing Python memory allocations
> +* PEP 3154, Pickle protocol revision 4
> +* PEP 3156, improved asynchronous IO support
> +
>  Other final large-scale changes:
>
>  * None so far
> @@ -85,12 +94,7 @@
>  * PEP 441, improved Python zip application support
>  * PEP 447, support for __locallookup__ metaclass method
>  * PEP 448, additional unpacking generalizations
> -* PEP 451, making custom import hooks easier to implement correctly
> -* PEP 453, pip bootstrapping/bundling with CPython
> -* PEP 454, PEP 445 based malloc tracing support
>  * PEP 455, key transforming dictionary
> -* PEP 3154, Pickle protocol revision 4
> -* PEP 3156, improved asynchronous IO support
>
>  Other proposed large-scale changes:
>
>
> --
> Repository URL: http://hg.python.org/peps
>
> ___
> Python-checkins mailing list
> python-check...@python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 454 - tracemalloc - accepted

[Charles-François Natali]

Hi,

I'm happy to officially accept PEP 454 aka tracemalloc.
The API has substantially improved over the past weeks, and is now
both easy to use and suitable as a fundation for high-level tools for
memory-profiling.

Thanks to Victor for his work!


Charles-François
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Heimes]

Am 21.11.2013 12:31, schrieb mar...@v.loewis.de:
> That sounds doable. If we provided a "python2.dll", would could make the
> header files using the "restricted API" by default if Python is compiled
> with VS 2010. Extension builders could then regularly compile their
> extensions with VS 2010, or VS 2013, despite Python itself being linked
> with the VS 2008 CRT.

What about the CAPI functions like PyFile_FromFile() and PyFile_AsFile()
that take a FILE* as argument? Or functions like PyErr_SetFromErrno()
that use the errno thread local variable? These function will either
crash or not work properly with mixed CRTs. Every CRT has its own errno
TLS, so Python won't see the errno of a CRT100 function like open(2),
see http://bugs.python.org/issue15883 .

I don't understand how a stable Python ABI solves the issue of unstable
CRT ABIs. Are you planning to address these issues?

Christian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Greg Ewing]

mar...@v.loewis.de wrote:

Package authors would have to create multiple binary releases of
the same modules for Windows, and upload them to PyPI. pip would have
to learn to download the right one, depending on what build of Python
2.7 is running.


Is that much different from package authors having to
release binaries for different versions of Python,
if they want to support older versions?

Having multiple binaries for the same x.y version
is different from what's been done before, but it
seems to me an unavoidable consequence of supporting
one x.y version for longer than usual.

If we *don't* provide a version for the current
MS compiler, we will end up with 2.7 becoming
effectively unsupported due to the actions of MS.
Do we want to allow that to happen, or do we
want 2.7 to remain viable for a while longer?

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Steve Dower]

Nick Coghlan wrote:
> For 2.7.7, I think some combination of the two following ideas would be worth
> pursuing:
> - a C runtime independent API flag (set by default on Windows when building 
> with
> a compiler other than VS2008). This would largely be a backport of some of the
> stable ABI work from Python 3.
> - getting Windows closer to the current Mac OS X situation by ensuring that 
> the
> C runtime used directly affects the ABI flags and shared library names. PyPI
> would apply the Mac OS X guideline where extensions are expected to be
> compatible with the python.org binaries.

I don't really think either of these are necessary. With some changes to 
Python's headers and some extra exports, it should be possible to future-proof 
Python 2.7.7 against any new compilers, at least on Windows.

What I have in mind is basically detecting the MSVC version in the headers 
(there are preprocessor variables for this) and, if it isn't VC9, substituting 
a different function for those that require FILE*. This function/macro could 
call _get_osfhandle() and pass it to an API (built into python27.dll) that 
calls _open_osfhandle() and forwards it to the usual API. 

This should let any compiler be used for building extensions or hosting 
python27.dll without affecting existing code or requiring changes to the 
packages.

> This would be the biggest change pushed through under the "make builds work"
> policy for the extended 2.7 lifecycle, but Microsoft's aggressive approach to
> deprecating old compilers and C runtimes means I think we don't have much
> choice.

Ultimately, compilers are probably going to be deprecated more quickly now that 
we're on a faster release cadence, which makes it more important that Python 
2.7 is prepared for an unknown future.

> In the near term, if Stackless build to a different DLL name under VS2010 and
> make it clear to their users that extension compatibility issues are possible
> (or even likely) if they aren't rebuilt from source, then I think that would 
> be
> compatible with the above proposal for a way forward.
> Then we'd just need some volunteers to write and implement a PEP or two :)

I'm happy to work on a PEP and changes for what I described above, if there's 
enough interest? I can also update distutils to detect and build with any 
available compiler, though this may be more of a feature than we'd want for 2.7 
at this point. 

Cheers,
Steve

> (Note, similar to the Mac OS X situation, I think we should do this without
> hosting any new interpreter variants on python.org - VS2010 and VS2013 source
> builds would become separate build-from-source ecosystems for extensions, 
> using
> sdists on PyPI as the default distribution mechanism)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Nick Coghlan]

On 21 November 2013 21:31,   wrote:
>
> Quoting Nick Coghlan :
>
>> Another alternative I'd prefer to an ABI version bump: backporting the "C
>> runtime independence" aspects of the stable ABI to Python 2.7.
>
> P.S. Thinking about this, there are some issues. The "restricted API"
> hides the object layout of all objects, in particular of type objects.
> Adding the PEP 384 API (PyType_FromSpec) might be a bit heavy for 2.7.
>
> So it might by better to provide a "py27compat.dll" instead which does
> not hide the structures (as they won't change during the remaining life
> of 2.7), but only hides any APIs and macros that directly expose CRT
> functionality.

Yep, that's what I meant by backporting just the "C runtime
independence" aspects - there's no reason to backport the version
independence features, since we're not planning to do another Python
2.x ABI bump anyway.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Kristján Valur Jónsson]

> > For Stackless, neither argument applies because 2.8 work would be done
> > by us and stackless has no particular allegiance towards either version.
> 
> Stackless can release their own Stackless 2.8 if they want, but I don't get 
> why
> CPython would have a 2.8 too.

Oh, this is the misunderstanding.  No one is trying to get permission for 
"CPython 2.8",
only "Stackless Python 2.8".

The "namespace" question from Christian has to do with a "python28.dll" which 
would be
built using VS2010, that this would never clash with a CPython version built the
same way.   such clashes would be very unfortunate.

Of course, we could even make a full break, if there will never be a CPython 
2.8 (which there won't be)
and call the dll slpython28.dll.

Cheers,

K


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Tismer]

On 21/11/13 22:13, Glenn Linderman wrote:

On 11/21/2013 12:23 PM, Christian Tismer wrote:

Maybe I would generate a cpython and spython exe and support them
both in the same distribution? 

That sounds cool, if possible.


Hooka Hooka!

Let's see if the nightmares agree :-)

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[martin]

Quoting Christian Tismer :


Can I rely on PEP 404 that the "Python 2.8" namespace never will clash
with CPython?


This question still hasn't been answered (AFAICT). So let me try to answer
it, and apologies upfront for being picky.

First, I don't understand the question: What is the "Python 2.8"
*namespace*? I try to rephrase the question:

# May I rely on PEP 404 that CPython will never make a Python 2.8 release?

To that, the clear answer is YES.

A natural follow-up question then is

# May I use the name "Python 2.8" for my own products?

To that, the answer is "ONLY IF YOU GET PERMISSION". This is really
off-topic for python-dev, but you would have to ask the PSF trademark
committee for permission to use that name as a product name. IIUC,
the current policy is that you may use "Python" if it a) is related
to the Python programming language, and b) is somehow qualified
(e.g. "Stackless Python"). To use just "Python", you need permission
(and I suspect that this might not be granted).

HTH,
Martin



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 10:53 AM, Christian Tismer wrote:

> I also think having a 2.8 out there that is exactly the same as 2.7,
>> except that it was built with a different version of a compiler on one
>> particular platform is a very very bad idea.
>>
> This was not my proposal. I was seeking a way to make a version that
> produces no collisions with DLLs, and it was a question about Stackless,
> not Python. But I admit that I was not aware of a license problem.
>
>
well, as you said below, you want to keep binary compatibility between
stackless and cPython, right down to the same dll name, so yes, it is about
Python. And since we are talking about it -- it actually would be nice to
be able to have a builds of python for Windows (any version) that are
not restricted to one compiler version per python version. (Like a VS2010
py2.7, for example, but not JUST that)


>> So if a different build of 2.7 for Windows is put out there, we need to
>> make sure it reports the platform in a way that wheel and pip can make the
>> distinction.
>>
>>
> That was the reason to change the number. If other approaches like a
> different
> name for certain things is easy to do, I am fine with that, too.


well, there is precedent for that with the OS-X builds -- so reasonable
enough. However, that really only helps for binary wheels and pip -- which
haven't been widely adopted yet (to put it mildly!). So maybe a new dll
name makes sense -- honestly  while some of how Windows handles dlls makes
"dll hell" inevitable, it's made worse by really short dll names, and
re-using names even when versions change -- we're so far past the 8.3
world, I have no idea why that's still done.

so a python27_VS_2010.dll or something might make sense.

Throw some info in there about 64 vs 32 bit too? or is it enough that the
linker will let you know if there is a problem?

It might be nice to patch the win_inst too--IIRC it's still not very smart
>> about even 32 vs 64 bit builds.
>>
>> As for stackless--just to be clear--can you use extensions built with the
>> "regular" python with a stack less binary? If so, I understand the concern.
>> If not, then it seems stackless is a separate ecosystem anyway.
>>
>
> Good question, and this _is_ a problem:
> Minus a few number of things, Stackless is almost completely binary
> compatible with extensions, and people _will_ want to try Stackless for
> some
> things or might want to go back and use CPython, be it only to remove
> concerns of
> a customer.
> People do want to install binary packages which are not built for
> Stackless,
> and we always did best efforts to support that.
>
> That means: basically we want to improve the situation for Stackless-based
> project in the first place, but make it easy to go back to CPython.
> We have a compiler switch that can completely remove the stackless
> additions and create a 100 % binary identical CPython version!
>
> That implies in the end that extension modules which work with Stackless
> will also be runnable on CPython 2.7.x VS2010 or whatever name it is.
> The naming problem then comes in again through the back-door,
> and we cannot shut our eyes and pretend "hey it is Stackless",
> because that is admittedly close to a fraud.
>
> So even if VS2010 exists only in the stackless branch, it is very likely
> to get used as CPython VS 2010, and I again have the naming problem ...
>

Just to be clear here:

You want to be able to create a non-stackless, regular old CPython binary
built with VS2010. (which is also compatible with  stackless build)

OK, now:

Do you want people to be able to use extensions built by third parties for
python.org CPython with your binary builds?

If so, then there will need to be a python.org binary built with VS2010,
and a way that makes it hard for people to try to use extensions built for
a different VS-version-build.

If not, then the only problem is that users of your VS2010-built binary
will go off willy nilly and try to use extensions built for
python.orgbuilds, and they may appear to work at first glance, but may
do weird
things or crash unexpectedly.

I'd say the issue there is education as much as anything.

Or couldn't you simply install your binary somewhere other than
C:\python27? (and use different registry setting or whatever so the windows
installers will not get confused?)

The other potential route for error is a pip install -- you don't want pip
to find a binary that is incompatible with your build -- so you should
assure that whatever pip/wheel uses to identify the build is
set differently in your build (see the relevant PEPs).

Note that for OS-X we have some of the same issues -- what with Homebrew
and Macports, and Apple, and ... there are a lot of potentially binary
incompatible builds of PythonX.Y out there. I don't think the issue really
is safely resolved, but at a policy level, I THINK the conclusion on the
distutils list was to declare that we only support binary wheels on PyPi
for the python.org bui

Re: [Python-Dev] PEP 0404 and VS 2010

[Brett Cannon]

On Thu, Nov 21, 2013 at 4:09 PM, Greg Ewing wrote:

> Concerning the version number, I thought the intention of
> PEP 404 was simply to say that the PSF would not be releasing
> anything called Python 2.8, not to forbid anyone *else*
> from doing so.
>
> Or am I wrong about that?
>

Well, it's essentially BSD open source. We can't stop anyone from doing
anything when it comes to code.


>
> If I'm right, there's nothing stopping Christian from
> releasing Stackless Python 2.8 with whatever improvements
> he wants.


You're right, there is nothing legally stopping him.


> If it includes other improvements beyond the
> change of compiler, it may even deserve the higher
> version number.


I disagree with that. Python-dev will not be releasing Python 2.8, ever
(that's what PEP 404 states). If I decided to start making decisions of
what constituted Python 2.8 outside of python-dev then it really isn't a
new feature release and thus does not deserve the number as it isn't a
release from the collective decision-making of the core developers of
Python. That's why so many people lept up initially when Christian
said  "Stackless
Python 2.8"; it isn't a true successor and no one wants to confuse users
into thinking that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Greg Ewing]

Kristján Valur Jónsson wrote:

The "namespace" question from Christian has to do with a "python28.dll" which 
would be
built using VS2010, that this would never clash with a CPython version built the
same way.


However, it *would* clash with someone else who did the same
thing, e.g. Fred Bloggs releases BloggPython 2.8 and calls
his dll python28.dll.

So it would be a bit rude to claim a bare pythonxy.dll
name for something that's not an official PSF version.

--
Greg


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] peps: Update list of accepted but not yet implemented or merged peps.

[Barry Warsaw]

On Nov 22, 2013, at 02:04 AM, Victor Stinner wrote:

>Hum, I don't think that regex module will enter Python 3.4 before this
>week-end, there is no PEP.

Okay thanks, I'll remove this from PEP 429.

>For the "Introspection information for builtins", I think the PEP 436
>has been accepted. The code has been merged, but the PEP status is
>still draft.

This is an odd one.  This is the only relevant message I could find.  Guido
only gives his approval to merge Argument Clinic in early:

https://mail.python.org/pipermail/python-dev/2013-October/129591.html

But this is not acceptance of PEP 436, which has still not gotten formal
approval afaict.  We'll need that in order to change the status on the PEP,
with a Resolution header.  Larry, is the PEP ready to be pronounced upon?

-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Tismer]

On 21/11/13 20:46, Chris Barker wrote:
well, as you said below, you want to keep binary compatibility between 
stackless and cPython, right down to the same dll name, so yes, it is 
about Python. And since we are talking about it -- it actually would 
be nice to be able to have a builds of python for Windows (any 
version) that are not restricted to one compiler version per python 
version. (Like a VS2010 py2.7, for example, but not JUST that)


That is a great target that I cannot address right now, but would love 
to work

on that, when I have understood all the API/ABI miracles. I was not aware of
those things that are already there.



well, there is precedent for that with the OS-X builds -- so 
reasonable enough. However, that really only helps for binary wheels 
and pip -- which haven't been widely adopted yet (to put it mildly!). 
So maybe a new dll name makes sense -- honestly  while some of how 
Windows handles dlls makes "dll hell" inevitable, it's made worse by 
really short dll names, and re-using names even when versions change 
-- we're so far past the 8.3 world, I have no idea why that's still done.


so a python27_VS_2010.dll or something might make sense.



I am converted to an OS X developer since 2006, but never had ABI problems,
because I use homebrew, but instead of being set free on Windows after 
30 years

of pain, I now have the same mess in my Parallels VMs.
Customers are so cruel, aren't they?

Throw some info in there about 64 vs 32 bit too? or is it enough that 
the linker will let you know if there is a problem?


It might be nice to patch the win_inst too--IIRC it's still
not very smart about even 32 vs 64 bit builds.

As for stackless--just to be clear--can you use extensions
built with the "regular" python with a stack less binary? If
so, I understand the concern. If not, then it seems stackless
is a separate ecosystem anyway.


Good question, and this _is_ a problem:
Minus a few number of things, Stackless is almost completely binary
compatible with extensions, and people _will_ want to try
Stackless for some
things or might want to go back and use CPython, be it only to
remove concerns of
a customer.
People do want to install binary packages which are not built for
Stackless,
and we always did best efforts to support that.

That means: basically we want to improve the situation for
Stackless-based
project in the first place, but make it easy to go back to CPython.
We have a compiler switch that can completely remove the stackless
additions and create a 100 % binary identical CPython version!

That implies in the end that extension modules which work with
Stackless
will also be runnable on CPython 2.7.x VS2010 or whatever name it is.
The naming problem then comes in again through the back-door,
and we cannot shut our eyes and pretend "hey it is Stackless",
because that is admittedly close to a fraud.

So even if VS2010 exists only in the stackless branch, it is very
likely
to get used as CPython VS 2010, and I again have the naming
problem ...


Just to be clear here:

You want to be able to create a non-stackless, regular old CPython 
binary built with VS2010. (which is also compatible with  stackless build)


Yes, this is the idea, to some contorted definition of 'idea'.


OK, now:

Do you want people to be able to use extensions built by third parties 
for python.org  CPython with your binary builds?


Would be great, but I would not mind to create their extensions on 
stackless.com, instead.




If so, then there will need to be a python.org  
binary built with VS2010, and a way that makes it hard for people to 
try to use extensions built for a different VS-version-build.


If not, then the only problem is that users of your VS2010-built 
binary will go off willy nilly and try to use extensions built for 
python.org  builds, and they may appear to work at 
first glance, but may do weird things or crash unexpectedly.


Yes, in the end it is much better to get some changes into CPython.
But as I read the input from Nick and Martin, I am afraid this is over
my tops, at least for the timeline I have set for myself.
(And yeah, I was pushy, as I always was with the Stackless project - 
forgive me).




I'd say the issue there is education as much as anything.

Or couldn't you simply install your binary somewhere other than 
C:\python27? (and use different registry setting or whatever so the 
windows installers will not get confused?)




Yes I can, and I'm pretty much considering. Seeking an improvement right 
now,

not a controversial path or whatnot...

The other potential route for error is a pip install -- you don't want 
pip to find a binary that is incompatible with your build -- so you 
should assure that whatever pip/wheel uses to identify the build is 
set diffe

Re: [Python-Dev] PEP 0404 and VS 2010

[martin]

Quoting Christian Heimes :


What about the CAPI functions like PyFile_FromFile() and PyFile_AsFile()
that take a FILE* as argument?


They are unable in the stable ABI, and would be unavailable in
py27compat.dll. Modules using them would have to be rewritten to not
use them anymore, or continue to be compiled with VS 2008.

Regards,
Martin


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] flaky tests caused by repr() sort order

[Christian Heimes]

Hi,

the buildbots are flaky because two repr() tests for userdict and
functools.partial fail every now and then. The test cases depend on a
fixed order of keyword arguments the representation of userdict and
partial instances. The improved hash randomization of PEP 456 shows its
power. I haven't seen the issue before because it happens rarely and
mostly on 32 bit platforms.

http://bugs.python.org/issue19681
http://bugs.python.org/issue19664

I'm not sure about the best approach for the issues. Either we need to
change the test case and make it more resilient or the code for repr()
must sort its dict keys.

Christian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 1:02 PM, Greg Ewing wrote:

> mar...@v.loewis.de wrote:
>
>> Package authors would have to create multiple binary releases of
>> the same modules for Windows, and upload them to PyPI. pip would have
>> to learn to download the right one, depending on what build of Python
>> 2.7 is running.
>>
>
> Is that much different from package authors having to
> release binaries for different versions of Python,
> if they want to support older versions?
>

or a better analogy -- both 32 and 64 bit versions -- same python version,
different binary compatibility

and really exactly like the OS-X situation: one for OS-X version 10.3+ (ppc
and intel32) and one for 10.6+(intel32+64).

In both cases, users need to know what they have installed, and be careful
that they use a matching binary extension when they grab one.

And we are in process of having PyPi and pip do the right thing with wheels.

I think the "only" added complication is that a 64bit cpython built with
VS2008 and and one built with VS2010 would look exactly the same to both
the user and the linker (i.e. python27.dll would be different, but still
link).

So it would be helpful if we could:

- Have a file naming scheme so people know what they are getting for both
the python binary and extension binaries.

 - Have a wheel naming and identifying scheme that also makes
that distinction.

 - Probably have a dll naming scheme that makes the distinction as well, so
if the above two checks fail, users will get a clear failure at runtime,
rather than a buggy success.



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker - NOAA Federal]

with older releases (I admit I don't understand the ABI compatibility on
OSX).


Well, with OS-X, it's not exactly the C lic in the same way, but there are
different SDKs for different OS versions, and you can add to that PPC vs
Intel processors and 32 vs 64 bit.

So we have for years had two builds for OS-X, and you can add to that
Macports, Homebrew, and what have you.

And the idea that this isn't an issue for gcc makes no sense-- it's such a
big issue for Linux, in fact that python.org doesn't even try to build
binaries for Linux, and pypi has disabled binary wheels for Linux.



So adding VS 2010 build files to the 2.7 tree would be fine with me,


I think this part is a no brainer.

I also think having a 2.8 out there that is exactly the same as 2.7, except
that it was built with a different version of a compiler on one particular
platform is a very very bad idea.

So how CAN this be addressed? This is really a distribution issue, and the
distutils-sig has been hard at work building the infrastructure to support
things like this.

Right now, if you build binary wheels with the two different "official"
OS-X builds, you will get two different names, and pop can find the correct
one. (last I checked, pip would happily install the wrong one if you asked
it to, though I think that is slated to be fixed)

So if a different build of 2.7 for Windows is put out there, we need to
make sure it reports the platform in a way that wheel and pip can make the
distinction.

It might be nice to patch the win_inst too--IIRC it's still not very smart
about even 32 vs 64 bit builds.

As for stackless--just to be clear--can you use extensions built with the
"regular" python with a stack less binary? If so, I understand the concern.
If not, then it seems stackless is a separate ecosystem anyway.

Chris
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Brett Cannon]

On Thu, Nov 21, 2013 at 10:31 AM, Christian Heimes wrote:

> Am 21.11.2013 16:12, schrieb Barry Warsaw:
> > On Nov 21, 2013, at 02:16 PM, Kristján Valur Jónsson wrote:
> >
> >> Oh, this is the misunderstanding.  No one is trying to get permission
> for
> >> "CPython 2.8", only "Stackless Python 2.8".
> >
> > I think this is a very bad idea.  We've worked hard to send the message
> that
> > the migration path is to Python 3 and while I understand that many people
> > don't want to or cannot switch to Python 3, backpeddling on this message
> from
> > the Python community will be bad PR and open the floodgates for pressure
> to
> > continue the Python 2 lineage.
> >
> > Stackless can of course do whatever it wants, but I would strongly
> oppose the
> > use of the word "Python" in that.  I don't want to be antagonistic, but I
> > think the PSF should also oppose allowing the use of the trademark for
> > any association with an unofficial 2.8.
>
> Yes, please don't use a name that contains both the strings "Python" and
> "2.8". It's going to create lots and lots of confusion. I strongly urge
> you to call it "Stackless 2.8" or something similar.
>

Agreed. PyPy has their own version number so there is precedent of keeping
a version number that is not directly tied to the version of Python that
one is compatible with. If you're really worried about confusion then go
the OS X route and call it "Stackless 10" and start way up the version
number chain where  there is no possible clash for several decades. =)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 1:12 PM, Paul Moore  wrote:

> None of the currently available binary distribution formats
>  distinguish Windows binaries by anything other than minor version. For
> wheels (and I think eggs), this is a showstopper as the name is
> essential metadata (compatibility tags) for the other formats (wininst
> and msi) the name is merely informational - packagers could rename,
> but (a) they will forget, and (b) the users won't know if they have or
> not.
>

exactly.


> Before we can cleanly support multiple ABIs for a single minor version
> on Windows, we need to have a resolution of this dilemma (which may be
> nothing more than "only binaries for the python.org builds are allowed
> on PyPI"...)


That's already the unstated case. But besides stackless, it some of us
are advocating that there be python.org-provided binaries built with a
newer compiler (eventually, anyway). Also, I haven't gotten a reply, but I
get the impression that Christian would like stackless-users not to have a
n easy way to get this all messed up.

the wheel namign scheme is defined by PEP 425. The bit in play here is:

"""
The platform tag is simply distutils.util.get_platform() with all hyphens -
and periods . replaced with underscore _.
win32
linux_i386
linux_x86_64
"""

I suspect that now we have only win32 and win64 for platform_tags for the
pyton.org Windows builds. But I'm also pretty sure that, for instance,
cygwin builds use a different tag.

And the "official" python.org OS-X builds have two different platform tags
for the two builds.

So the precedent is there -- and it's easy enough to keep "win32" as the
VS2008 version, and then have a "win32_VS_2010" or whatever for a newer
build.

That wouldn't take much to do, and it would allow pip and binary wheels to
"just work".

It would be nice if the msi installers could be similarly patched, but I
have no idea what that would take.

-Chris











>  Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] flaky tests caused by repr() sort order

[Guido van Rossum]

Correct.
On Nov 21, 2013 10:15 AM, "Daniel Holth"  wrote:

> +1 on unsorted repr(). It makes it obvious that the collection is not
> sorted.
>
> On Thu, Nov 21, 2013 at 1:10 PM, Christian Heimes 
> wrote:
> > Am 21.11.2013 18:57, schrieb Tim Peters:
> >> Best to change the failing tests.  For example, _they_ can sort the
> >> dict keys if they rely on a fixed order.  Sorting in general is a
> >> dubious idea because it can be a major expense with no real benefit
> >> for most uses.
> >
> > I don't consider repr() as a performance critical function. It's mostly
> > used for debugging.
> >
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] flaky tests caused by repr() sort order

[Nick Coghlan]

On 22 Nov 2013 04:12, "Christian Heimes"  wrote:
>
> Am 21.11.2013 18:57, schrieb Tim Peters:
> > Best to change the failing tests.  For example, _they_ can sort the
> > dict keys if they rely on a fixed order.  Sorting in general is a
> > dubious idea because it can be a major expense with no real benefit
> > for most uses.
>
> I don't consider repr() as a performance critical function. It's mostly
> used for debugging.

+1 from me for fixing the tests (likely by checking the expected sort order
rather than assuming it) rather than changing the repr() implementations.

Cheers,
Nick.

>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Christian Tismer]

...

I also think having a 2.8 out there that is exactly the same as 2.7, 
except that it was built with a different version of a compiler on one 
particular platform is a very very bad idea.




This was not my proposal. I was seeking a way to make a version that
produces no collisions with DLLs, and it was a question about Stackless,
not Python. But I admit that I was not aware of a license problem.

So how CAN this be addressed? This is really a distribution issue, and 
the distutils-sig has been hard at work building the infrastructure to 
support things like this.


Right now, if you build binary wheels with the two different 
"official" OS-X builds, you will get two different names, and pop can 
find the correct one. (last I checked, pip would happily install the 
wrong one if you asked it to, though I think that is slated to be fixed)


So if a different build of 2.7 for Windows is put out there, we need 
to make sure it reports the platform in a way that wheel and pip can 
make the distinction.




That was the reason to change the number. If other approaches like a 
different

name for certain things is easy to do, I am fine with that, too.

It might be nice to patch the win_inst too--IIRC it's still not very 
smart about even 32 vs 64 bit builds.


As for stackless--just to be clear--can you use extensions built with 
the "regular" python with a stack less binary? If so, I understand the 
concern. If not, then it seems stackless is a separate ecosystem anyway.


Good question, and this _is_ a problem:
Minus a few number of things, Stackless is almost completely binary
compatible with extensions, and people _will_ want to try Stackless for some
things or might want to go back and use CPython, be it only to remove 
concerns of

a customer.
People do want to install binary packages which are not built for Stackless,
and we always did best efforts to support that.

That means: basically we want to improve the situation for Stackless-based
project in the first place, but make it easy to go back to CPython.
We have a compiler switch that can completely remove the stackless
additions and create a 100 % binary identical CPython version!

That implies in the end that extension modules which work with Stackless
will also be runnable on CPython 2.7.x VS2010 or whatever name it is.
The naming problem then comes in again through the back-door,
and we cannot shut our eyes and pretend "hey it is Stackless",
because that is admittedly close to a fraud.

So even if VS2010 exists only in the stackless branch, it is very likely
to get used as CPython VS 2010, and I again have the naming problem ...

cheers - Chris

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Greg Ewing]

Concerning the version number, I thought the intention of
PEP 404 was simply to say that the PSF would not be releasing
anything called Python 2.8, not to forbid anyone *else*
from doing so.

Or am I wrong about that?

If I'm right, there's nothing stopping Christian from
releasing Stackless Python 2.8 with whatever improvements
he wants. If it includes other improvements beyond the
change of compiler, it may even deserve the higher
version number.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0404 and VS 2010

[Chris Barker]

On Thu, Nov 21, 2013 at 2:53 PM, Nick Coghlan  wrote:

> For 2.7.7, I think some combination of the two following ideas would be
> worth pursuing:
>
> - a C runtime independent API flag (set by default on Windows when
> building with a compiler other than VS2008). This would largely be a
> backport of some of the stable ABI work from Python 3.
>
very cool!

> - getting Windows closer to the current Mac OS X situation by ensuring
> that the C runtime used directly affects the ABI flags and shared library
> names. PyPI would apply the Mac OS X guideline where extensions are
> expected to be compatible with the python.org binaries.
>
sounds good.

> (Note, similar to the Mac OS X situation, I think we should do this
> without hosting any new interpreter variants on python.org -
>
but we do host different variants for OS-X on python.org. It's complicated
by the universal binary thing, but there are two variants hosted. There are
also two Windows variants  32 and 64 bit -- yes you can think of those as
different platforms, but as you can run the 32 bit build on a 64 bit
Windows, things do get confused -- I've seen a lot of questions on mailing
lists that have to do with those two getting mixed up (the msi installer
really could be smarter...)

I'm not really proposing that python.org host another varient at this
point, but I dont hink it should be off the table.

> VS2010 and VS2013 source builds would become separate build-from-source
> ecosystems for extensions, using sdists on PyPI as the default distribution
> mechanism)
>
A lot of Windows users need binaries -- it would be really nice if we could
make that easy. But the key thing is that people who do a "pip install
something" should not get an incompatible binary. And the way to do that is
make sure the wheel naming scheme used by binary builds in teh wild don't
use the same naming scheme as pyton.org builds if they  are not compatible
with them.

As it sounds like stackless is going to do this soon, it woulld be nice to
have some consensus as to what the standard should be.

By the way, I'm still not sure if it should go in the ABI tag or the
Platfrom tag. from the examples in the PEP, it doesn't look like it should
be the ABI tab, and in the case of teh OS-X builds, it's in the platfrm tag.

Also, on my OS-X box, with a somewhat hacked ptyon,org build, I get:

pyGnome-alpha-cp27-none-macosx_10_6_i386.whl

when I build a binary wheel.

Note that the "abi" tag is "none" -- not sure why that is, this is clealy
cpython2.7.* -- shouldn't that be cp27 for the ABI tag?

The wheel docs are kind of sparse, so I have no idea where that abi tag is
supposed to come from -- ah, I found something in PEP 425:

"""Why is the ABI tag (the second tag) sometimes "none" in the reference
implementation? Since Python 2 does not have an easy way to get to the
SOABI (the concept comes from newer versions of Python 3) the reference
implentation at the time of writing guesses "none". Ideally it would detect
"py27(d|m|u)" analagous to newer versions of Python, but in the meantime
"none" is a good enough way to say "don't know".
"""
I don't know what SOABI is, but it soudns like that defines what should be
in the abi tag...


But the platfrom tag is:
  macosx_10_6_i386.whl

macosx -- natch'
10_6 -- built for the 10.6 SDK (I'm running 10.7...)
i386 -- I've hacked my build to be 32 bit only

Does this belong on the distutils list instead?

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com