Re: Does Apache Beam for python support server-based shuffle with Dataflow runner yet?

2018-01-17 Thread Ahmet Altay
Hi Nima, You can try this feature with python SDK using the same instructions from the announcement. However, it is not ready for production usage. Team is working official supporting it. We cannot share an ETA, once it is available it will be announced. For future questions related to the

Does Apache Beam for python support server-based shuffle with Dataflow runner yet?

2018-01-17 Thread Nima Mousavi
Hi, In June 2017, Google introduced server-based shuffle for Datatflow pipeline, which can result in 5x performance improvement. However, at the time of announcement this feature was only available for Cloud Dataflow SDK for Java version 1. What is the status for Dataflow SDK for Python? Is it