If we want all our code related to pipeline execution to be in a container,
what value does building wheel distributions provide?
On Mon, Feb 12, 2018 at 1:18 PM, Kenneth Knowles <k...@google.com> wrote:
> On Mon, Feb 12, 2018 at 1:04 PM, Charles Chen <c...@google.com> wrote:
>> Currently, Apache Beam distributes Python packages through pip and PyPI.
>> On PyPI, developers can release either source tarballs, and / or
>> precompiled "wheel" distributions for each platform, which would be used if
>> available for a particular platform. Currently, we only distribute the
>> source tarballs, so any user who installs Beam using "pip install
>> apache_beam" has to have a compiler and toolchain installed to take
>> advantage of Cython optimizations in Beam (which require compiled C code).
>> If such a compiler is not available, Beam is currently configured to
>> install anyway, but will use slower Python codepaths instead of the more
>> optimized ones (for example, for Coder encoding / decoding).
>> I would like to propose that we start distributing binary wheel
>> distributions for our releases, for common platforms like Windows / Mac /
>> Linux. We could potentially use a method similar to this one (
>> https://github.com/MacPython/cython-wheels) for building these wheel
>> distributions. Thoughts?