+1 for the proposal :) I think this will fit nicely with the current API.
If we use deployments we could simply extend the current status interfaces with taskManagerDeploymentStatus (we already have jobManagerDeploymentStatus) for the standalone mode. We would have to check how we could use the Zookeeper HA to implement last-state, there is quite a bit of trickery involved in the current implementation that is specific to k8s ha. Gyula On Tue, Apr 26, 2022 at 1:23 PM Jassat, Usamah <us...@amazon.co.uk.invalid> wrote: > Thanks for the feedback. > > # The TaskManager replicas > Yeah I think this makes sense and explicitly stating TM replicas makes > more sense in the standalone mode. I will update the FLIP and clarify this. > > # How the JobManager and TaskManager pods are managed? > I think deployments for both TaskManager and JobManager pods should be > sufficient as we just need a pool of TaskManagers and this also matches the > Flink documentation for setting up a Flink Standalone cluster on k8s [1]. > > > # Version support > That's great to hear that we can support 1.13. > > I would want to support ZooKeeper HA for last-state upgrade however if > this is some sort of limitation that doesn't make this possible then we can > fall-back to limitations for older versions. Do you know if there are any > limitations of ZooKeeper HA that may mean it won't work with last-state > upgrade mode? > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/ > > Thanks, > Usamah > > On 26/04/2022, 10:10, "Yang Wang" <danrtsey...@gmail.com> wrote: > > CAUTION: This email originated from outside of the organization. Do > not click links or open attachments unless you can confirm the sender and > know the content is safe. > > > > Thanks for creating the FLIP-223 and starting the discussion. > > I have some quick questions. > > # The TaskManager replicas > > > The TaskManager replicas need to be configured both for standalone > session > and application. Because it could not be calculated if the parallelism > is > set via java codes. > > > # How the JobManager and TaskManager pods are managed? > > We could use k8s Deployment to manage the JobManager pods. Of cause, > k8s > Job, StatefulSet also make sense. > > > What would you like to do for the TaskManager pods? > > > > # Version support > > Native support could work from 1.13 and I have created a ticket for > this[1]. > > > Considering the last-state upgrade mode, the K8s HA should be enabled. > I am > afraid even standalone mode before 1.12 could not work. > > Do you want to introduce the ZooKeeper HA or add some limitations for > version choice? > > > > > [1]. https://issues.apache.org/jira/browse/FLINK-27412 > > > > Best, > > Yang > > Jassat, Usamah <us...@amazon.co.uk.invalid> 于2022年4月25日周一 18:25写道： > > > Hi everyone, > > > > We would like to start the discussion of the adding standalone mode > > support to the Flink Kubernetes operator. Standalone mode was > initially > > considered as part of FLIP-212 but decided to be out of scope to > focus on > > Flink native k8s integration for that FLIP [1]. Standalone support > will > > also open the door to supporting previous Flink versions in the > operator > > which I would also like to open discussion about. > > > > I have created a FLIP with the details on the general changes that > we are > > proposing: > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-223%3A+Implement+standalone+mode+support+in+the+kubernetes+operator > > > > > > Looking forward to your feedback. > > > > Regards, > > Usamah > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > > > >
