On Tue, 4 Sep 2018 at 20:30, Nathaniel Smith <n...@pobox.com> wrote: > > On Mon, Sep 3, 2018 at 4:51 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > > On Mon., 3 Sep. 2018, 5:48 am Ronald Oussoren, <ronaldousso...@mac.com> > > wrote: > >> > >> > >> What’s the problem with including GPU and non-GPU variants of code in a > >> binary wheel other than the size of the wheel? I tend to prefer binaries > >> that work “everywhere", even if that requires some more work in building > >> binaries (such as including multiple variants of extensions to have > >> optimised code for different CPU variants, such as SSE and non-SSE variants > >> in the past). > > > > > > As far as I'm aware, binary artifact size *is* the problem. It's just that > > once you're automatically building and pushing an artifact (or an image > > containing that artifact) to thousands or tens of thousands of managed > > systems, the wasted bandwidth from pushing redundant implementations of the > > same functionality becomes more of a concern than the convenience of being > > able to use the same artifact across multiple platforms. > > None of the links that Dustin gave at the top of the thread are about > managed systems though.
When you're only managing a few systems, or only saving a few MB per download, "install both and pick at runtime" is an entirely viable option. However, since tensorflow is the example, neither of those cases is true: 1. It's a Google project, so they have tens of thousands of instances to worry about (as do other cloud providers) 2. The size difference is in the tens or even hundreds of megabytes I didn't actually realise the GPU tensorflow package was over 200 MB, though - that's large enough to be noticeably slow to extract and install even from a local zipfile, let alone if you're needing to download it first. Cheers, Nick. P.S. If anyone were to be worrying about this problem in the context of PyPI specifically, the most likely candidates would actually be Fastly, and then to a lesser degree, AWS, as they're handling the bulk of any data transfer costs associated with folks downloading packages that are larger than they really need to be :) -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/AJI7KENXH6ISHNSRVYAFDW4QK2BAHO77/
