On 25 August 2017 at 09:38, Thomas Kluyver <tho...@kluyver.me.uk> wrote: > On Fri, Aug 25, 2017, at 08:47 AM, Paul Moore wrote: >> I've never cared about whether the backend modifies the source tree, >> only that the direct-wheel / sdist->wheel equivalence holds. > > If a user types 'pip install .' in a directory where they don't have > write permissions, what should happen? > > 1. The backend is capable of building without writing in that directory. > 2. The frontend (pip) notices and pre-emptively copies to a writable tmp > directory. > 3. The user accepts that the operation may fail, depending on the > backend. > > I'd go for option 1 or 3.
I'd go for option (1). I don't think we want to get into (2), with the frontend trying to spot possible issues. I think (3) is always a possibility, but users have a right to expect that backends will do whatever they can to avoid it. I'd *also* expect that the wheel built (and hence what gets installed) would be the same as what you'd get if the user added write permissions and then ran the command. That's part of what I mean by direct wheel and sdist->wheel being equivalent. I'd completely support any user who complained that getting different results in those 2 cases is a bug in the backend. But as Nick pointed out, this is probably more detail than we need in the PEP. > Here's a proposed update to the build_wheel section of the PEP: (Apologies if I miss a change here, I don't have the original copy right now to compare against). > build_wheel(wheel_directory, config_settings=None, > metadata_directory=None): > ... > > Must build a .whl file, and place it in the specified wheel_directory. > It must return the basename (not the full path) of the .whl file it > creates, as a unicode string. > > If the build frontend has previously called > prepare_metadata_for_build_wheel and depends on the wheel resulting from > this call to have metadata matching this earlier call, then it should > provide the path to the created .dist-info directory as the > metadata_directory argument. If this argument is provided, then > build_wheel MUST produce a wheel with identical metadata. The directory > passed in by the build frontend MUST be identical to the directory > created by prepare_metadata_for_build_wheel, including any unrecognized > files it created. > > Backends which do not provide the prepare_metadata_for_build_wheel hook > may either silently ignore the metadata_directory parameter to > build_wheel, or else raise an exception when it is set to anything other > than None. > > Wheels built by this hook should, as far as possible, be equivalent to > first building an sdist and then building a wheel from that sdist. This is the point that I care about. > Frontends which wish to ensure this consistency may call build_sdist > (see below) first, and then call build_wheel in the unpacked sdist. But > if build_sdist fails with NotImplementedError, they will fall back to > calling build_wheel in the source directory, and rely on the backend to > produce an equivalent wheel. This seems to contradict the previous point - I'd rephrase it as "Frontends which want to protect against backends violating this expectation may call build_sdist..." and at that point it starts to seem like excessive paranoia on the part of frontends. I repeat my earlier comment here - do we have any actual cases where we expect in-place wheel builds to *not* enforce the equivalence? This was originally a huge debate issue because of all the comments about development builds and incremental compilation, and caching. Are the people who insisted that backends be allowed to carry build artifacts between compiles now accepting that they have to ensure build consistency? Or have they just dropped out of the discussion? I'm pretty sure Nathaniel was one of those people, but I'm equally sure he's fed up of debating theoretical possibilities. I suspect he still considers my assumptions fairly naive, though, as I have no experience of the sort of complexities involved in building the scientific stack... Either way, pip *can* go sdist->wheel, but as things are worded here, I don't see a real reason to bother. I'd be more inclined to push any bug reports about getting different results to the backend, quoting the PEP's requirement for build equivalence, and only implement sdist->wheel in pip if the backends pushed back (at which point, we'd need to reopen the debate on what the statement in the PEP about equivalence actually meant... ) > The source directory may be read-only. Backends should therefore be > prepared to build without creating or modifying any files in the source > directory, but they may opt not to handle this case, in which case > failures will be visible to the user. Frontends are not responsible for > any special handling of read-only source directories. This is a useful addition, particularly as it's a general point, not tied to any specific backend implementation. Personally, I'd expect a backend to do its best to avoid failing - for example by copying the files to a temporary location and doing a build there - but that's reasonably considered a backend quality of implementation issue. > The backend may store intermediate artifacts in cache locations or > temporary directories. The presence or absence of any caches should not > make a material difference to the final result of the build. I'm fine with this point. I'd also be fine with noting that such cache locations can be stored in the source tree if the backend wishes to do so (in which case, the sdist->wheel route is effectively a way of deleting any such caches). But that's probably unnecessary. Overall, this looks good to me. Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
