Yes. On Wed, Sep 11, 2019 at 3:12 AM Andrea Medeghini < [email protected]> wrote:
> Hi Lukasz, > > Thanks for your answer. > > I had a look at the ProcessEnvironmentFactory, and I understand that the > factory is expecting to run the boot process which is in the python > container (source file boot.go). Is that right? > > > > Best, > > Andrea > > > > *From: *Lukasz Cwik <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Tuesday, 10 September 2019 at 23:19 > *To: *user <[email protected]> > *Subject: *Re: How do I run Beam Python pipelines using Flink deployed on > Kubernetes? > > > > [CAUTION - EXTERNAL EMAIL] > > External environments are mainly used for testing since they represent > environments that are already running. There are other clever uses of it as > well but unlikely what your looking for. > > > > The docker way will be nice once there is integration between the > FlinkRunner and its supported cluster managers to automatically manage the > SDK container instances. Until that exists you could look at using the > process environment (it launches the SDK as a child process beside the > Flink workers). > > > > On Tue, Sep 10, 2019 at 4:38 AM Andrea Medeghini < > [email protected]> wrote: > > Hello, > > Can I run Beam Python pipelines with Flink deployed on Kubernetes? > > > > I have successfully managed to run a Beam Python pipeline using the > Portable runner and the job service pointing to a local Flink server > running as Docker containers. > > I was able to achieve that mounting the Docker socket in my Flink > container and running Flink as root, so the class DockerEnvironmentFactory > can create the Python harness container. > > Unfortunately, I can't use the same solution when Flink is running in > Kubernetes. Moreover, I don't want to create the Python harness container > using the Docker command from my pods. > > I noticed there is an implementation called ExternalEnvironmentFactory, > but I am not sure how to use it. > > Is there a way to deploy a side container and use a different factory to > run the Python harness process? > > What is the correct approach? > > > > Thanks, > > Andrea > > > > > > > > [image: Image removed by sender. 10x] <https://www.10xbanking.com/> > > > 10x Banking > > Please consider the environment before printing this e-mail > > ********************************************************************************************************************************************************** > This e-mail, including any attachments, is confidential. If you are not > the intended recipient, you may not copy, forward, disclose or use any part > of it. If you have received this e-mail in error, please notify the sender > immediately by return e-mail and delete this message and any copies and > attachments. > > 10x Future Technologies Services Limited is registered in England & Wales > Registered Number: 11262659 > Registered Address: 5 Howick Place, Victoria, London, SW1P 1WG > > > > [image: 10x] <https://www.10xbanking.com/> > > 10x Banking > > Please consider the environment before printing this e-mail > > ********************************************************************************************************************************************************** > This e-mail, including any attachments, is confidential. If you are not > the intended recipient, you may not copy, forward, disclose or use any part > of it. If you have received this e-mail in error, please notify the sender > immediately by return e-mail and delete this message and any copies and > attachments. > > 10x Future Technologies Services Limited is registered in England & Wales > Registered Number: 11262659 > Registered Address: 5 Howick Place, Victoria, London, SW1P 1WG >
