Re: Deploying native Kubernetes via yaml files?
Hi Niels, I have created a ticket[1] to track the yaml file submission for native K8s integration. Feel free to share your significative thoughts about this way. [1]. https://issues.apache.org/jira/browse/FLINK-16760 Best, Yang Niels Basjes 于2020年3月24日周二 下午11:10写道： > Thanks. > I'll have a look at this. > > Can you post the ticket number for this feature (after you created it) > please? > > Niels > > On Tue, Mar 24, 2020 at 3:54 PM Yang Wang wrote: > >> Hi Niels, >> >> Currently, the native integration Flink cluster could not be created via >> yaml file. The reason >> why we introduce the native integration is for the users who are not >> familiar with K8s and >> kubectl. So we want to make it easier for our Flink users to deploy Flink >> cluster on K8s. >> >> However, i think some other users, especially K8s experts, they really >> like the yaml way to >> create Flink cluster. I will create a ticket for this feature. If you >> want to do this manually now, >> i think you need to at least do the following changes. >> 1. Update the `flink-console.sh` and add a new script(mostly like >> jobmanager.sh) to set the >> entrypoint to >> "org.apache.flink.kubernetes.entrypoint.KubernetesSessionClusterEntrypoint". >> 2. Create the configmap, service like the standalone on K8s[1]. >> 3. Create a jobmanager.yaml with service account(enough permission to >> create pod in K8s >> cluster) correctly set >> >> >> [1]. >> https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html#appendix >> >> >> Best, >> Yang >> >> Ufuk Celebi 于2020年3月24日周二 下午9:51写道： >> >>> PS: See also >>> https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html#flink-job-cluster-on-kubernetes >>> >>> On Tue, Mar 24, 2020 at 2:49 PM Ufuk Celebi wrote: >>> Hey Niels, you can check out the README with example configuration files here: https://github.com/apache/flink/tree/master/flink-container/kubernetes Is that what you were looking for? Best, Ufuk On Tue, Mar 24, 2020 at 2:42 PM Niels Basjes wrote: > Hi, > > As clearly documented here > https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html > the > current way of deploying Flink natively on Kubernetes is by running the > ./bin/kubernetes-session.sh > script that runs some Java code that does "magic" to deploy in on the > cluster. > This works. > > I was wondering: Is it with the current code base already possible to > craft a set of Yaml files (perhaps even with a special Docker image) so > that I can deploy it using the 'normal' Kubernetes way of doing a kubectl > apply -f foo.yaml ? > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes > > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes >
Re: Deploying native Kubernetes via yaml files?
Hi Niels, Currently, the native integration Flink cluster could not be created via yaml file. The reason why we introduce the native integration is for the users who are not familiar with K8s and kubectl. So we want to make it easier for our Flink users to deploy Flink cluster on K8s. However, i think some other users, especially K8s experts, they really like the yaml way to create Flink cluster. I will create a ticket for this feature. If you want to do this manually now, i think you need to at least do the following changes. 1. Update the `flink-console.sh` and add a new script(mostly like jobmanager.sh) to set the entrypoint to "org.apache.flink.kubernetes.entrypoint.KubernetesSessionClusterEntrypoint". 2. Create the configmap, service like the standalone on K8s[1]. 3. Create a jobmanager.yaml with service account(enough permission to create pod in K8s cluster) correctly set [1]. https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html#appendix Best, Yang Ufuk Celebi 于2020年3月24日周二 下午9:51写道： > PS: See also > https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html#flink-job-cluster-on-kubernetes > > On Tue, Mar 24, 2020 at 2:49 PM Ufuk Celebi wrote: > >> Hey Niels, >> >> you can check out the README with example configuration files here: >> https://github.com/apache/flink/tree/master/flink-container/kubernetes >> >> Is that what you were looking for? >> >> Best, >> >> Ufuk >> >> On Tue, Mar 24, 2020 at 2:42 PM Niels Basjes wrote: >> >>> Hi, >>> >>> As clearly documented here >>> https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html >>> the >>> current way of deploying Flink natively on Kubernetes is by running the >>> ./bin/kubernetes-session.sh >>> script that runs some Java code that does "magic" to deploy in on the >>> cluster. >>> This works. >>> >>> I was wondering: Is it with the current code base already possible to >>> craft a set of Yaml files (perhaps even with a special Docker image) so >>> that I can deploy it using the 'normal' Kubernetes way of doing a kubectl >>> apply -f foo.yaml ? >>> >>> -- >>> Best regards / Met vriendelijke groeten, >>> >>> Niels Basjes >>> >>
Re: Deploying native Kubernetes via yaml files?
PS: See also https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html#flink-job-cluster-on-kubernetes On Tue, Mar 24, 2020 at 2:49 PM Ufuk Celebi wrote: > Hey Niels, > > you can check out the README with example configuration files here: > https://github.com/apache/flink/tree/master/flink-container/kubernetes > > Is that what you were looking for? > > Best, > > Ufuk > > On Tue, Mar 24, 2020 at 2:42 PM Niels Basjes wrote: > >> Hi, >> >> As clearly documented here >> https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html >> the >> current way of deploying Flink natively on Kubernetes is by running the >> ./bin/kubernetes-session.sh >> script that runs some Java code that does "magic" to deploy in on the >> cluster. >> This works. >> >> I was wondering: Is it with the current code base already possible to >> craft a set of Yaml files (perhaps even with a special Docker image) so >> that I can deploy it using the 'normal' Kubernetes way of doing a kubectl >> apply -f foo.yaml ? >> >> -- >> Best regards / Met vriendelijke groeten, >> >> Niels Basjes >> >
Re: Deploying native Kubernetes via yaml files?
Hey Niels, you can check out the README with example configuration files here: https://github.com/apache/flink/tree/master/flink-container/kubernetes Is that what you were looking for? Best, Ufuk On Tue, Mar 24, 2020 at 2:42 PM Niels Basjes wrote: > Hi, > > As clearly documented here > https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html > the > current way of deploying Flink natively on Kubernetes is by running the > ./bin/kubernetes-session.sh > script that runs some Java code that does "magic" to deploy in on the > cluster. > This works. > > I was wondering: Is it with the current code base already possible to > craft a set of Yaml files (perhaps even with a special Docker image) so > that I can deploy it using the 'normal' Kubernetes way of doing a kubectl > apply -f foo.yaml ? > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes >
Deploying native Kubernetes via yaml files?
Hi, As clearly documented here https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html the current way of deploying Flink natively on Kubernetes is by running the ./bin/kubernetes-session.sh script that runs some Java code that does "magic" to deploy in on the cluster. This works. I was wondering: Is it with the current code base already possible to craft a set of Yaml files (perhaps even with a special Docker image) so that I can deploy it using the 'normal' Kubernetes way of doing a kubectl apply -f foo.yaml ? -- Best regards / Met vriendelijke groeten, Niels Basjes
