Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
Thanks for the feedback.
I updated the PR to use get_platform and get_host_platform.
More testing is still needed before it's ready to merge to make sure it still
does what it was intended to do.
-Original Message-
From: Python-Dev On
Behalf Of Xavier de Gaye
Sent: Tuesday, February 19, 2019 12:06 PM
To: Steve Dower ; python-dev@python.org
Subject: Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the
"target" platform
> [Any reason for dropping python-dev?]
Sorry. just clicked the wrong button.
> And the answer is a resounding "yes, it returns the target platform"? It
> seems you're saying this, but the wording of your email sounds just enough of
> a question that I'm not sure whether you are definitively answering it or not.
The answer is yes indeed, it returns the target platform.
This is a listing of the non-obvious steps that lead to this conclusion.
Xavier
On 2/19/19 8:45 PM, Steve Dower wrote:
> [Any reason for dropping python-dev?]
>
> On 19Feb2019 1139, Xavier de Gaye wrote:
>> Is distutils.util.get_platform() the "current" or the "target" platform ?
>
> I *think* you're answering this below, yes?
>
>> * When cross-compiling on posix platforms using autoconf, configure is
>> run with the '--host=host-type' [1] command line option to specify
>> the target platform.
>>
>> * The AC_CANONICAL_HOST macro is used by configure.ac to get the
>> canonical variables `host' and `host_cpu' [2].
>> Those variables are used to compute _PYTHON_HOST_PLATFORM.
>>
>> * The Makefile generated by configure runs setup.py,
>> generate-posix-vars, etc... on the build platform using
>> PYTHON_FOR_BUILD, a native python interpreter that is set
>> to run with the _PYTHON_HOST_PLATFORM environment variable.
>>
>> * get_platform() in setup.py and in Lib/distutils/util.py returns the
>> value of _PYTHON_HOST_PLATFORM when cross-compiling.
>>
>> So the process of cross-compilation on posix platforms has
>> get_platform() return the target ('host' in autoconf terminology) platform.
>>
>> [1]
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Fsavannah-checkouts%2Fgnu%2Fautoconf%2Fmanual%2Fautoconf-2.69%2Fhtml_node%2FSpecifying-Target-Triplets.htmldata=02%7C01%7CPaul.Monson%40microsoft.com%7C186de203c3d644c7517208d696a5f266%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636862036817328765sdata=2PMKn%2Bt5ed82gkSBnew0nT1TA1qzDuU3VZNOFPYPqIM%3Dreserved=0
>> [2]
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Fsavannah-checkouts%2Fgnu%2Fautoconf%2Fmanual%2Fautoconf-2.69%2Fhtml_node%2FCanonicalizing.htmldata=02%7C01%7CPaul.Monson%40microsoft.com%7C186de203c3d644c7517208d696a5f266%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636862036817328765sdata=zRtcyLjDrkL%2BfcignLCDjtUri29j7mCscdVkGqhNk50%3Dreserved=0
>
> And the answer is a resounding "yes, it returns the target platform"? It
> seems you're saying this, but the wording of your email sounds just enough of
> a question that I'm not sure whether you are definitively answering it or not.
>
> Cheers,
> Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fpython-devdata=02%7C01%7CPaul.Monson%40microsoft.com%7C186de203c3d644c7517208d696a5f266%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636862036817328765sdata=Au0xrggEwNRRAen6dr8GnBuTTvMbxVhSuWOtX67p3Ts%3Dreserved=0
Unsubscribe:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Foptions%2Fpython-dev%2Fpaulmon%2540microsoft.comdata=02%7C01%7CPaul.Monson%40microsoft.com%7C186de203c3d644c7517208d696a5f266%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636862036817328765sdata=nVHiEW69so29h6uTbY7XDjPF6%2FwU0pnlNpQEeYSoygY%3Dreserved=0
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
[Any reason for dropping python-dev?]
Sorry. just clicked the wrong button.
And the answer is a resounding "yes, it returns the target platform"? It seems
you're saying this, but the wording of your email sounds just enough of a question that
I'm not sure whether you are definitively answering it or not.
The answer is yes indeed, it returns the target platform.
This is a listing of the non-obvious steps that lead to this conclusion.
Xavier
On 2/19/19 8:45 PM, Steve Dower wrote:
[Any reason for dropping python-dev?]
On 19Feb2019 1139, Xavier de Gaye wrote:
Is distutils.util.get_platform() the "current" or the "target" platform ?
I *think* you're answering this below, yes?
* When cross-compiling on posix platforms using autoconf, configure is
run with the '--host=host-type' [1] command line option to specify
the target platform.
* The AC_CANONICAL_HOST macro is used by configure.ac to get the
canonical variables `host' and `host_cpu' [2].
Those variables are used to compute _PYTHON_HOST_PLATFORM.
* The Makefile generated by configure runs setup.py,
generate-posix-vars, etc... on the build platform using
PYTHON_FOR_BUILD, a native python interpreter that is set
to run with the _PYTHON_HOST_PLATFORM environment variable.
* get_platform() in setup.py and in Lib/distutils/util.py returns the
value of _PYTHON_HOST_PLATFORM when cross-compiling.
So the process of cross-compilation on posix platforms has
get_platform() return the target ('host' in autoconf terminology) platform.
[1]
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Specifying-Target-Triplets.html
[2]
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Canonicalizing.html
And the answer is a resounding "yes, it returns the target platform"? It seems
you're saying this, but the wording of your email sounds just enough of a question that
I'm not sure whether you are definitively answering it or not.
Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
On 16Feb.2019 0831, Nick Coghlan wrote: > On Sat, 16 Feb 2019 at 08:06, Steve Dower wrote: >> I'm inclined to say that nobody but us uses this API :) Does that make >> it seem more okay to "clarify" that it's returning target platform? > > I've always treated the situation as "Cross-compilation doesn't work, > build on the target platform, using a VM if you have to", and I > suspect a lot of folks have approached the status quo the same way. For platforms where pyconfig.h is generated, this is still going to be true, at least until the compiler classes learn to add a platform-specific include path. On Windows, we have a static pyconfig.h that changes behaviour based on compiler and Windows SDK provided preprocessor directives, so we can quite comfortably use the same file. > So if there are functions you can change to make cross-compilation > actually work without requiring changes to a lot of other projects, > that seems like a good thing to me. Okay Paul (Monson), that's your cue to update the PR :) Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
On Sat, 16 Feb 2019 at 08:06, Steve Dower wrote: > I'm inclined to say that nobody but us uses this API :) Does that make > it seem more okay to "clarify" that it's returning target platform? I've always treated the situation as "Cross-compilation doesn't work, build on the target platform, using a VM if you have to", and I suspect a lot of folks have approached the status quo the same way. So if there are functions you can change to make cross-compilation actually work without requiring changes to a lot of other projects, that seems like a good thing to me. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
On Fri, Feb 15, 2019 at 2:02 PM Steve Dower wrote: > On 14Feb2019 1147, Gregory P. Smith wrote: > > To alleviate confusion long term I'd love it if we could deprecate the > > unqualified get_platform() API and point people towards always being > > explicit about get_target_platform() vs get_current_platform(). > > This is an option too, though it doesn't reduce the code churn. I > personally want to consider distutils deprecated as a whole anyway, and > only maintained for the sake of our core needs. > > > There are valid reasons for people to be expecting either target or > > current return values from get_platform(), but I agree with you, having > > it return the target platform /feels/ more likely to be what people > > want. It'd be worth auditing a random sample of people's calls of this > > API in open source projects to confirm that intuition. > > I took a random sample of about 50 uses from GitHub and 100% of them > were copies of our distutils/tests/test_util.py (not even kidding: > https://github.com/search?q=distutils+get_platform=Code) > > If you go far enough down the results, they're all copies of wheel's (or > pip's) pep425tags.py, which import distutils.util but don't seem to use > get_platform > > I'm inclined to say that nobody but us uses this API :) Does that make > it seem more okay to "clarify" that it's returning target platform? > All of the instances of its use that I can find in a quick search (excluding copies/clones/forks of other code) are using it to mean target platform as well. So yeah, I'd just go with that assumption. -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
On 14Feb2019 1147, Gregory P. Smith wrote: To alleviate confusion long term I'd love it if we could deprecate the unqualified get_platform() API and point people towards always being explicit about get_target_platform() vs get_current_platform(). This is an option too, though it doesn't reduce the code churn. I personally want to consider distutils deprecated as a whole anyway, and only maintained for the sake of our core needs. There are valid reasons for people to be expecting either target or current return values from get_platform(), but I agree with you, having it return the target platform /feels/ more likely to be what people want. It'd be worth auditing a random sample of people's calls of this API in open source projects to confirm that intuition. I took a random sample of about 50 uses from GitHub and 100% of them were copies of our distutils/tests/test_util.py (not even kidding: https://github.com/search?q=distutils+get_platform=Code) If you go far enough down the results, they're all copies of wheel's (or pip's) pep425tags.py, which import distutils.util but don't seem to use get_platform I'm inclined to say that nobody but us uses this API :) Does that make it seem more okay to "clarify" that it's returning target platform? Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
On Thu, Feb 14, 2019 at 11:38 AM Steve Dower wrote: > As part of adding ARM32 support for Windows, we need to enable > cross-compilation in distutils. This is easy enough, though it requires > somehow getting the target platform as well as the current platform. > > Right now, the change at https://github.com/python/cpython/pull/11774 > adds a get_target_platform() function for this and updates (as far as I > can tell) all uses of get_platform() to use this instead. I would rather > just change get_platform() to return the target platform. > > The current docs are somewhat vague on exactly what this function does, > and I suspect have been mostly written from an "always build from > source" mentality that may have implied, but not explicitly considered > cross-compilation. > > https://docs.python.org/3/distutils/apiref.html#distutils.util.get_platform > > "Return a string that identifies the current platform. This is used > mainly to distinguish platform-specific build directories and > platform-specific built distributions." > > So it says "current" platform, explicitly says that "os.uname()" is the > source, but then goes on to say: > > "For non-POSIX platforms, currently just returns sys.platform." > > Which is incorrect, as sys.platform is always "win32" always but > get_platform() already returns "win-amd64" for 64-bit builds. > > And also: > > "For Mac OS X systems the OS version reflects the minimal version on > which binaries will run (that is, the value of MACOSX_DEPLOYMENT_TARGET > during the build of Python), not the OS version of the current system." > > So it seems like this function is already returning "the default > platform that should be used when building extensions" - ignoring bugs > in libraries that monkeypatch distutils, the "--plat-name" option should > entirely override the return value of this function. > > Given this, does it seem to be okay to have it determine and return the > target platform rather than the host platform? Right now, that would > only affect the new target of building for win-arm32, but I would also > like to update the documentation to make it more about how this value > should be used rather than where it comes from. > > Any objections or concerns? > To alleviate confusion long term I'd love it if we could deprecate the unqualified get_platform() API and point people towards always being explicit about get_target_platform() vs get_current_platform(). There are valid reasons for people to be expecting either target or current return values from get_platform(), but I agree with you, having it return the target platform *feels* more likely to be what people want. It'd be worth auditing a random sample of people's calls of this API in open source projects to confirm that intuition. -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
As part of adding ARM32 support for Windows, we need to enable cross-compilation in distutils. This is easy enough, though it requires somehow getting the target platform as well as the current platform. Right now, the change at https://github.com/python/cpython/pull/11774 adds a get_target_platform() function for this and updates (as far as I can tell) all uses of get_platform() to use this instead. I would rather just change get_platform() to return the target platform. The current docs are somewhat vague on exactly what this function does, and I suspect have been mostly written from an "always build from source" mentality that may have implied, but not explicitly considered cross-compilation. https://docs.python.org/3/distutils/apiref.html#distutils.util.get_platform "Return a string that identifies the current platform. This is used mainly to distinguish platform-specific build directories and platform-specific built distributions." So it says "current" platform, explicitly says that "os.uname()" is the source, but then goes on to say: "For non-POSIX platforms, currently just returns sys.platform." Which is incorrect, as sys.platform is always "win32" always but get_platform() already returns "win-amd64" for 64-bit builds. And also: "For Mac OS X systems the OS version reflects the minimal version on which binaries will run (that is, the value of MACOSX_DEPLOYMENT_TARGET during the build of Python), not the OS version of the current system." So it seems like this function is already returning "the default platform that should be used when building extensions" - ignoring bugs in libraries that monkeypatch distutils, the "--plat-name" option should entirely override the return value of this function. Given this, does it seem to be okay to have it determine and return the target platform rather than the host platform? Right now, that would only affect the new target of building for win-arm32, but I would also like to update the documentation to make it more about how this value should be used rather than where it comes from. Any objections or concerns? Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
