That's a good idea. Probably best to add an example in:
https://github.com/lyft/flinkk8soperator
Do you want to add an issue?
(It will have to wait for 2.18 release though.)
On Fri, Nov 1, 2019 at 11:37 AM Chad Dombrova wrote:
> Hi Thomas,
> Do you have an example Dockerfile demonstrating
Hi Thomas,
Do you have an example Dockerfile demonstrating best practices for building
an image that contains both Flink and Beam SDK dependencies? That would be
useful.
-chad
On Fri, Nov 1, 2019 at 10:18 AM Thomas Weise wrote:
> For folks looking to run Beam on Flink on k8s, see update in
For folks looking to run Beam on Flink on k8s, see update in [1]
I also updated [2]
TLDR: at this time best option to run portable pipelines on k8s is to
create container images that have both Flink and the SDK dependencies.
I'm curious how much interest there is to use the official SDK
The changes to containerize the Python SDK worker pool are nearly complete.
I also updated the document for next implementation steps.
The favored approach (initially targeted option) for pipeline submission is
support for the (externally created) fat far. It will keep changes to the
operator to
Hi Thomas,
Nice work! It's really clearly presented.
What's the current favored approach for pipeline submission?
I'm also interested to know how this plan overlaps (if at all) with the
work on Fine-Grained Resource Scheduling [1][2] that's being done for Flink
1.9+, which has implications for
I am very happy to see this. I'll take a look, and leave my comments.
I think this is something we'd been needing, and it's great that you guys
are putting thought into it. Thanks!<3
On Wed, Jul 24, 2019 at 9:01 PM Thomas Weise wrote:
> Hi,
>
> Recently Lyft open sourced *FlinkK8sOperator,* a
Hi,
Recently Lyft open sourced *FlinkK8sOperator,* a Kubernetes operator to
manage Flink deployments on Kubernetes:
https://github.com/lyft/flinkk8soperator/
We are now discussing how to extend this operator to also support
deployment of Python Beam pipelines with the Flink runner. I would like
