Re: Proposal: build Python wheel distributions for Apache Beam releases

I have rarely* "pointed at a tarball and installed" but used `pip install ` which I understand is implemented as you describe, but with some naive version logic. Maybe just personal style, I just can't go back from package management, even if it lacks real dependency resolution. I do know that pip

Re: Proposal: build Python wheel distributions for Apache Beam releases

Python development isn't generally like Java in this sense--you just point people at a tarball that they install. Granted, this doesn't work as well when one has a set of 100 tarballs that all should be installed together, but again that's not the Python way to release a project. I did find

Re: Proposal: build Python wheel distributions for Apache Beam releases

I think the thing is just that you can point a user env at it to test like a user. Like you can do this with https://repository.apache.org/content/repositories/orgapachebeam-1030/ but not https://dist.apache.org/repos/dist/dev/beam/2.4.0/ On Wed, Mar 7, 2018 at 5:03 PM Ahmet Altay

Re: Proposal: build Python wheel distributions for Apache Beam releases

I do not know what is the best practice. For practical purposes it makes sense to stage to the same svn repo, so that we can test it as part of the release process. On Wed, Mar 7, 2018 at 4:22 PM, Robert Bradshaw wrote: > Yes, we should. There's a bit of an open question of

Re: Proposal: build Python wheel distributions for Apache Beam releases

Yes, we should. There's a bit of an open question of where these release artifacts should be staged. (Eventually, of course, they'll be published to PyPi). Should they be placed alongside the source artifacts in the svn repository? On Wed, Mar 7, 2018 at 3:00 PM Ahmet Altay

Re: Proposal: build Python wheel distributions for Apache Beam releases

Are we planning to do this for the 2.4.0 release? I am asking, because they were not part of RC1 artifacts. On Tue, Feb 13, 2018 at 9:18 AM, Robert Bradshaw wrote: > On Tue, Feb 13, 2018 at 8:31 AM, Nima Mousavi > wrote: > > Related question: > > >

Re: Proposal: build Python wheel distributions for Apache Beam releases

On Tue, Feb 13, 2018 at 8:31 AM, Nima Mousavi wrote: > Related question: > > How can we tell if the docker image of our binary contains the cython > optimized beam or the slower codepath? > The image was built on Google cloud (using gcloud container builds submit). There

Re: Proposal: build Python wheel distributions for Apache Beam releases

Related question: How can we tell if the docker image of our binary contains the cython optimized beam or the slower codepath? The image was built on Google cloud (using *gcloud container builds submit* ). On Mon, Feb 12, 2018 at 9:32 PM, Ahmet Altay wrote: > +1 to wheels.

Re: Proposal: build Python wheel distributions for Apache Beam releases

+1 to wheels. The main effort for this would be updating the release guide, and adding support for other platforms in Jenkins for building and testing wheels. In light of this, maybe we can prioritize having test infrastructure for other platforms. On Mon, Feb 12, 2018 at 1:47 PM, Ismaël Mejía

Re: Proposal: build Python wheel distributions for Apache Beam releases

+1 for wheels, they are the standard binary distribution format so it makes sense. Also wheels support packaging python 2 and 3 on universal packages so they are future proof. On Mon, Feb 12, 2018 at 10:26 PM, Robert Bradshaw wrote: > +1, is it too late to try to release

Re: Proposal: build Python wheel distributions for Apache Beam releases

+1, is it too late to try to release these as part of the 2.3 release (to get familiar with the process, no code changes should be needed)? The wheels are advantageous when running locally (e.g. during testing and development) where requiring containers will probably be overkill. This will become

Re: Proposal: build Python wheel distributions for Apache Beam releases

If we want all our code related to pipeline execution to be in a container, what value does building wheel distributions provide? On Mon, Feb 12, 2018 at 1:18 PM, Kenneth Knowles wrote: > +1 > > On Mon, Feb 12, 2018 at 1:04 PM, Charles Chen wrote: > >>

Re: Proposal: build Python wheel distributions for Apache Beam releases

+1 On Mon, Feb 12, 2018 at 1:04 PM, Charles Chen wrote: > Currently, Apache Beam distributes Python packages through pip and PyPI. > On PyPI, developers can release either source tarballs, and / or > precompiled "wheel" distributions for each platform, which would be used if >

Proposal: build Python wheel distributions for Apache Beam releases

Currently, Apache Beam distributes Python packages through pip and PyPI. On PyPI, developers can release either source tarballs, and / or precompiled "wheel" distributions for each platform, which would be used if available for a particular platform. Currently, we only distribute the source