Does Apache Beam for python support server-based shuffle with Dataflow runner yet?

2018-01-17 Thread Nima Mousavi
Hi, In June 2017, Google introduced server-based shuffle for Datatflow pipeline, which can result in 5x performance improvement. However, at the time of announcement this feature was only available for Cloud Dataflow SDK for Java version 1. What is the status for Dataflow SDK for Python? Is it

Re: Proposal: build Python wheel distributions for Apache Beam releases

2018-02-13 Thread Nima Mousavi
Related question: How can we tell if the docker image of our binary contains the cython optimized beam or the slower codepath? The image was built on Google cloud (using *gcloud container builds submit* ). On Mon, Feb 12, 2018 at 9:32 PM, Ahmet Altay wrote: > +1 to wheels.