> On 2 Apr 2018, at 08:35, Joshua Root <[email protected]> wrote: > > Good summary from Rainer, which covers most of what I was going to say. > > On 2018-4-1 12:53 , Rainer Müller wrote: >> I see two reasons why python modules need natively compiled binary files: >> >> a) for performance >> b) they link against another library >> >> In case of b), using pre-compiled binary files is out of the question, >> because we want to link to our own libraries and not against libraries >> in /usr/lib. > > I'll just add that some extensions use libraries that do not ship with > the OS at all, which makes binary wheels impossible unless you either > statically link or just assume the user has the libs installed in some > preselected place (which will probably be /usr/local/lib). > > The direction among packagers in general seems to be towards encouraging > upstream authors to make it easy (or at least possible) for others to > reproduce their build process. See <https://reproducible-builds.org/>. > > - Josh
Thanks everybody for sharing your point of view. I was not aware of all the subtleties and your insights have been very helpful. Rainer: > In case of b), using pre-compiled binary files is out of the question, > because we want to link to our own libraries and not against libraries > in /usr/lib. Thanks for you detailed answer. Linked libraries are indeed an issue. > Can we rely on Portfile authors to check what is linked? > Are they aware which libraries this python module links to before > submitting the Portfile? I wondered whether a Portfile author should check and I recognise it's a brittle process at best. > It would probably always be safe to use wheels for platform "any", but > there is also no advantage over setup.py for them. I agree. > We would also lose +universal variants for python ports, but I did not > check if we actually have such ports. Thanks for pointing this out, I wasn't fully aware. > Whether we can > safely ship pre-compiled Java code is still not clear to me. I had quite a few surprises in the past while I was maintaining Logstash/ElasticSearch for FreeBSD or a running Jenkins on Solaris. These project (used to?) include native libraries which sometimes broke on certain OS versions. As far as I can tell, though, if it's pure Java byte code meant to run on a compliant runtime, then I would say it's safe. Ryan: > For what macOS versions and architectures are they compiled? I assume the > list is smaller than what MacPorts currently supports. The list is actually rather small: * tensorflow-1.6.0-...-macosx_10_11_x86_64.whl And for py-grpcio pretty much the same: * grpcio-1.10.0-...-macosx_10_11_x86_64.whl tensorflow-1.6.0-cp27-cp27m-macosx_10_11_x86_64.whl > An additional question is: can you fully trust the binary code of all > the hundreds of python packages? (It's also true that you should not > always trust the sources until you checked them, but binaries are > easier to infect without noticing.) Thanks Mojca, that's a very good point. Joshua: > The direction among packagers in general seems to be towards encouraging > upstream authors to make it easy (or at least possible) for others to > reproduce their build process. See <https://reproducible-builds.org/>. Thanks Joshua. Yes, that a great initiative. I think you've given me enough reasons to try and build everything from source. Thanks, -- Enrico
