Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-28 Thread Thomas Kluyver
On Thu, May 25, 2017, at 03:38 PM, Nick Coghlan wrote: > Seeing it like this pushes me from "Eh, maybe?" to "No, definitely not" > [on the log directory] :) That's fine by me. It does feel like unwanted extra complexity for both backends and frontends. And backends dealing with output in an

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Steve Dower
On 25May2017 0756, Paul Moore wrote: On 25 May 2017 at 15:38, Nick Coghlan wrote: So I'm inclined to accept the encoding amendment, and then provisionally accept the overall PEP pending implementation in pip. Me too. (Assuming I understand Steve's comments on backends,

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Paul Moore
On 25 May 2017 at 15:38, Nick Coghlan wrote: > Seeing it like this pushes me from "Eh, maybe?" to "No, definitely not" :) Agreed. Given that it's stated as optional for frontends to support it, I'd be arguing against pip bothering (as it seems like too much complexity) - so

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Nick Coghlan
On 26 May 2017 at 00:04, Thomas Kluyver wrote: > On Thu, May 25, 2017, at 02:27 PM, Paul Moore wrote: >> I'd be concerned here that we risk making the frontend UI a lot more >> complex for little actual benefit. I'd rather we stick with the >> current model, where a backend

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Thomas Kluyver
On Thu, May 25, 2017, at 02:27 PM, Paul Moore wrote: > I'd be concerned here that we risk making the frontend UI a lot more > complex for little actual benefit. I'd rather we stick with the > current model, where a backend just has some output to pass through to > the frontend. Let's get a

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Paul Moore
On 25 May 2017 at 13:26, Nick Coghlan wrote: > On 24 May 2017 at 20:29, Thomas Kluyver wrote: >> Nick: >>> That's actually pretty similar to the way tools like mock (the chroot >>> based RPM builder) work. That way, build backends could choose >>>

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Wayne Werner
FWIW, I was just reading an article about writing libraries to just operate on streams and totally ignore stdout/stdin/file io, and just leave the IO to something else. It may be a good idea to define the spec as purely operating on byte and text streams, then leave where those streams go as an

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-25 Thread Nick Coghlan
On 24 May 2017 at 20:29, Thomas Kluyver wrote: > Nick: >> That's actually pretty similar to the way tools like mock (the chroot >> based RPM builder) work. That way, build backends could choose >> between: >> >> - use pipes to stream output from the tools they call, deal

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-24 Thread Thomas Kluyver
On Wed, May 24, 2017, at 01:22 AM, Chris Jerdonek wrote: > 1) Would it make sense to provide a way for build tools to specify > what encoding they use (e.g. if not using the default), instead of > changing their encoding to conform to a standard? It seems like that > could be easier, although I

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-24 Thread Nick Coghlan
On 24 May 2017 at 03:04, Thomas Kluyver wrote: > I'll propose a variant of an idea I described already: the frontend > could provide the backend with a fresh temp directory. If the backend > needs to run other processes, it can redirect the output into a file in > that temp

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Chris Jerdonek
A couple comments: 1) Would it make sense to provide a way for build tools to specify what encoding they use (e.g. if not using the default), instead of changing their encoding to conform to a standard? It seems like that could be easier, although I know this doesn't address problems like

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Thomas Kluyver
On Tue, May 23, 2017, at 04:20 PM, Nick Coghlan wrote: > Up to this point, I've been in favour of both 1b and 2b, since they're Noted. > However, I also realised that there's a potential third way to handle > this problem: design a Python level API that allows front ends to use > more structured

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 17:16, Nick Coghlan wrote: > Yep, and that's also why I want to avoid trying to use it to improve > the encoding handling situation - pip and other tools have to deal > with the current mess regardless, and there's already likely to be > some significant

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Nick Coghlan
On 24 May 2017 at 01:39, Paul Moore wrote: > On 23 May 2017 at 16:20, Nick Coghlan wrote: >> Taking that approach of just defining a helper API and expecting build >> backends to either use it or emulate it gives us some quite attractive >> properties: >

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 16:20, Nick Coghlan wrote: > Taking that approach of just defining a helper API and expecting build > backends to either use it or emulate it gives us some quite attractive > properties: Making the output data part of a structured API (and by implication,

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Nick Coghlan
On 23 May 2017 at 22:41, Thomas Kluyver wrote: > On Tue, May 23, 2017, at 12:56 PM, Paul Moore wrote: > Can I take a quick poll of what people following this topic think? > > Q1: Default encoding for captured build stdout/stderr > a. UTF-8 (consistent, can represent any

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 13:41, Thomas Kluyver wrote: > Can I take a quick poll of what people following this topic think? > > Q1: Default encoding for captured build stdout/stderr > a. UTF-8 (consistent, can represent any character) > b. Locale default (convenient if backend runs

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Thomas Kluyver
On Tue, May 23, 2017, at 12:56 PM, Paul Moore wrote: > So based on your proposal, won't you introduce similar bugs by using > print() without sorting out encodings? Unless (see below) you assume > that the frontend sorts it out for you. If you strictly follow the locale encoding, you need to sort

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 12:36, Thomas Kluyver wrote: > As you described earlier, though, even using a locale dependent encoding > doesn't really avoid this problem, because of tools using OEM vs ANSI > codepages on Windows. And if PYTHONIOENCODING is set, Python processes > will

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Thomas Kluyver
On Tue, May 23, 2017, at 11:04 AM, Paul Moore wrote: > However, if we do this then we have a situation where existing build > tools (compilers, etc) that we have to support still use platform > dependent encodings. That's a reality that we can't wish away. And the > majority of real-life issues

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 09:56, Thomas Kluyver wrote: > I may have missed it, but has anyone proposed what it should do if it > wants to send characters which can't be encoded in the locale encoding? No, it's not been mentioned - the focus has been on running build tools like a

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Thomas Kluyver
On Tue, May 23, 2017, at 09:08 AM, Paul Moore wrote: > I strongly > prefer using the locale encoding as the assumed encoding for the > output stream rather than UTF-8. I may have missed it, but has anyone proposed what it should do if it wants to send characters which can't be encoded in the

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-23 Thread Paul Moore
On 23 May 2017 at 05:11, Nick Coghlan wrote: > What we can then also do is to recommend that *front-ends* do the > following when invoking their build backend CLI shims: > > 1. Implement the C locale -> UTF-8 based locale coercion defined in > PEP 538 when launching the

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Nick Coghlan
On 23 May 2017 at 03:38, Steve Dower wrote: > Okay, I think I get the problem now. We expect backends to let child > subprocesses just spit out whatever *they* want onto the same stdout/stderr. > > I'm really not a fan of forcing front ends to clean up that mess, and so

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Thomas Kluyver
On Mon, May 22, 2017, at 11:36 PM, Steve Dower wrote: > IMHO, #2 is definitely the right way to go. Yes, the platform specific > code now has to worry about the encoding, but... the encoding is > platform specific? So... that seems exactly right? :) Maybe I'm still > missing something here, but

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Steve Dower
On 22May2017 1253, Paul Moore wrote: It seems to me there are 2 schools of thought: 1. There are likely to be fewer front ends than back ends, and so the front end(s) (basically, pip) should deal with the problem. Also, backends are more likely to be written by developers who are looking at

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Paul Moore
On 22 May 2017 at 18:38, Steve Dower wrote: > Okay, I think I get the problem now. We expect backends to let child > subprocesses just spit out whatever *they* want onto the same stdout/stderr. s/expect/allow/ The paranoid in me suspects "expect" is also true, though :-)

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Paul Moore
On 22 May 2017 at 15:23, Nick Coghlan wrote: > No, that's discussed here: > https://www.python.org/dev/peps/pep-0517/#comparison-to-competing-proposals > > Even though PEP 517 defines a Python API for build backends to > implement, it still expects installation tools to wrap a

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Nick Coghlan
On 22 May 2017 at 23:15, Paul Moore wrote: > On 22 May 2017 at 12:28, Thomas Kluyver wrote: >> What if it wants to send a character which can't be encoded in the >> locale encoding? It's quite easy on Windows to end up with a character >> that you can't

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Paul Moore
On 22 May 2017 at 12:28, Thomas Kluyver wrote: > What if it wants to send a character which can't be encoded in the > locale encoding? It's quite easy on Windows to end up with a character > that you can't encode as cp1252. If the build tool uses .encode(loc_enc, >

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Nick Coghlan
On 22 May 2017 at 21:28, Thomas Kluyver wrote: > On Mon, May 22, 2017, at 12:02 PM, Paul Moore wrote: >> The only reservation I have is that the choice of UTF-8 means that on >> Windows, build backends pretty much have to explicitly manage tool >> output (as they are pretty

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Nick Coghlan
On 22 May 2017 at 21:02, Paul Moore wrote: > On 22 May 2017 at 11:22, Thomas Kluyver wrote: >> I have made a PR against the PEP with my best take on the encoding >> situation: >> https://github.com/python/peps/pull/264/files > > LGTM. > > The only

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Thomas Kluyver
On Mon, May 22, 2017, at 12:02 PM, Paul Moore wrote: > The only reservation I have is that the choice of UTF-8 means that on > Windows, build backends pretty much have to explicitly manage tool > output (as they are pretty much certain *not* to output in UTF-8). > Build backend writers that aren't

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Paul Moore
On 22 May 2017 at 11:22, Thomas Kluyver wrote: > I have made a PR against the PEP with my best take on the encoding > situation: > https://github.com/python/peps/pull/264/files LGTM. The only reservation I have is that the choice of UTF-8 means that on Windows, build

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Thomas Kluyver
I have made a PR against the PEP with my best take on the encoding situation: https://github.com/python/peps/pull/264/files On Mon, May 22, 2017, at 11:19 AM, Paul Moore wrote: > On 22 May 2017 at 10:56, Thomas Kluyver wrote: > > On Sat, May 20, 2017, at 07:36 PM, Steve

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Paul Moore
On 22 May 2017 at 10:56, Thomas Kluyver wrote: > On Sat, May 20, 2017, at 07:36 PM, Steve Dower wrote: >> Require that build tools either send UTF-8 to the UI component, or write >> bytes to a file and call it a build output. I see no benefit in >> requiring both the build

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-22 Thread Thomas Kluyver
On Sat, May 20, 2017, at 07:36 PM, Steve Dower wrote: > Require that build tools either send UTF-8 to the UI component, or write > bytes to a file and call it a build output. I see no benefit in > requiring both the build tool and the UI tool to guess what the text > encoding is. I'm not

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Nick Coghlan
On 21 May 2017 at 02:36, Steve Dower wrote: > On 20May2017 0820, Nick Coghlan wrote: >> >> Good point regarding the fact that the Windows 16-bit APIs only come >> into play for interactive sessions (even in 3.6+), while for PEP 517 >> we're specifically interested in the

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Donald Stufft
> On May 20, 2017, at 4:05 PM, Paul Moore wrote: > > I'm a little concerned if we're going to end up with a proposal that > means that distutils is in violation of the spec unless this issue is > fixed. I'm not sure if that's where we're headed, but I wanted to be > clear

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Steve Dower
On 20May2017 1315, Paul Moore wrote: On 20 May 2017 at 17:36, Steve Dower wrote: In general, since most subprocesses (at least on Windows) do not have customizable encodings, the tool that launches them needs to know what the encoding is. Since we don't live in a Python

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Paul Moore
On 20 May 2017 at 17:36, Steve Dower wrote: > In general, since most subprocesses (at least on Windows) do not have > customizable encodings, the tool that launches them needs to know what the > encoding is. Since we don't live in a Python 3.6 world quite yet, that means >

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Paul Moore
On 20 May 2017 at 19:36, Steve Dower wrote: > >> - As a lazy developer, I don't want to read stdout/stderr from a >> subprocess only to spit it back to my own stdout/stderr. I'd much rather >> just launch the subprocess and let it use the same stdout/stderr as my >> build

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Steve Dower
On 20May2017 1011, Thomas Kluyver wrote: On Sat, May 20, 2017, at 05:36 PM, Steve Dower wrote: In general, since most subprocesses (at least on Windows) do not have customizable encodings, the tool that launches them needs to know what the encoding is. Since we don't live in a Python 3.6 world

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Thomas Kluyver
On Sat, May 20, 2017, at 05:36 PM, Steve Dower wrote: > I'll probably read the PEP closely and see that this is entirely > incorrect, but if it's right: > > * encoding for text between the build UI and build tool should just be > specified once for all platforms (i.e. use UTF-8). +1 > *

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Steve Dower
On 20May2017 0820, Nick Coghlan wrote: Good point regarding the fact that the Windows 16-bit APIs only come into play for interactive sessions (even in 3.6+), while for PEP 517 we're specifically interested in the 8-bit pipes used to communicate with build subprocesses launched by an

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Brett Cannon
On Fri, May 19, 2017, 09:20 Thomas Kluyver, wrote: > On Fri, May 19, 2017, at 05:17 PM, Paul Moore wrote: > > On 19 May 2017 at 16:53, Daniel Holth wrote: > > > Congrats on getting 518 in. > > > > Agreed, by the way. That's a big step! > > Thanks both. It

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Nick Coghlan
Good point regarding the fact that the Windows 16-bit APIs only come into play for interactive sessions (even in 3.6+), while for PEP 517 we're specifically interested in the 8-bit pipes used to communicate with build subprocesses launched by an installation tool. On 20 May 2017 at 19:11, Paul

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Paul Moore
On 20 May 2017 at 09:03, Thomas Kluyver wrote: > On Sat, May 20, 2017, at 07:54 AM, Nick Coghlan wrote: >> * on platforms with 8-bit standard streams (e.g. Linux, Mac OS X), >> build systems SHOULD emit UTF-8 encoded output >> * on platforms with 16-bit standard streams

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Thomas Kluyver
On Sat, May 20, 2017, at 07:54 AM, Nick Coghlan wrote: > * on platforms with 8-bit standard streams (e.g. Linux, Mac OS X), > build systems SHOULD emit UTF-8 encoded output > * on platforms with 16-bit standard streams (e.g. Windows), build > systems SHOULD emit UTF-16-LE encoded output I'm quite

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Nick Coghlan
On 20 May 2017 at 01:16, Thomas Kluyver wrote: > On Fri, May 19, 2017, at 03:41 PM, Paul Moore wrote: >> Can we specify what encoding the informational text must be written >> in? > > Sure, that makes sense. What about: > > All hooks are run with working directory set to the

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-20 Thread Nick Coghlan
On 20 May 2017 at 00:18, Thomas Kluyver wrote: > Hi, > > I'd like to make another push for PEP 517, which would make it possible > to build wheels from a source tree with other build tools, without > needing setup.py. > > https://www.python.org/dev/peps/pep-0517/ > > Last

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
On Fri, May 19, 2017, at 05:17 PM, Paul Moore wrote: > On 19 May 2017 at 16:53, Daniel Holth wrote: > > Congrats on getting 518 in. > > Agreed, by the way. That's a big step! Thanks both. It does feel like an achievement. :-) ___

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Paul Moore
On 19 May 2017 at 16:53, Daniel Holth wrote: > Congrats on getting 518 in. Agreed, by the way. That's a big step! Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Daniel Holth
Congrats on getting 518 in. On Fri, May 19, 2017, 11:37 Thomas Kluyver wrote: > On Fri, May 19, 2017, at 04:31 PM, Paul Moore wrote: > > For flit, would having the install tool set PYTHONIOENCODING help? > > If install tools were meant to set PYTHONIOENCODING, I probably

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
On Fri, May 19, 2017, at 04:31 PM, Paul Moore wrote: > For flit, would having the install tool set PYTHONIOENCODING help? If install tools were meant to set PYTHONIOENCODING, I probably wouldn't do anything else in flit's code. Python should then take care of ensuring that any output is UTF-8

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Paul Moore
On 19 May 2017 at 16:16, Thomas Kluyver wrote: > On Fri, May 19, 2017, at 03:41 PM, Paul Moore wrote: >> Can we specify what encoding the informational text must be written >> in? > > Sure, that makes sense. What about: > > All hooks are run with working directory set to the

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
On Fri, May 19, 2017, at 03:41 PM, Paul Moore wrote: > Can we specify what encoding the informational text must be written > in? Sure, that makes sense. What about: All hooks are run with working directory set to the root of the source tree, and MAY print arbitrary informational text on stdout

Re: [Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Paul Moore
On 19 May 2017 at 15:18, Thomas Kluyver wrote: > Hi, > > I'd like to make another push for PEP 517, which would make it possible > to build wheels from a source tree with other build tools, without > needing setup.py. A point that came up recently while dealing with a pip

[Distutils] PEP 517 - specifying build system in pyproject.toml

2017-05-19 Thread Thomas Kluyver
Hi, I'd like to make another push for PEP 517, which would make it possible to build wheels from a source tree with other build tools, without needing setup.py. https://www.python.org/dev/peps/pep-0517/ Last time this was discussed we made a couple of minor changes to the PEP, but we didn't