Re: Native Kubernetes Task Managers

2024-06-09 Thread Alexis Sarda-Espinosa
dId` describes the pod index. > > > > Regards, > Zhuo. > Replied Message > | From | Alexis Sarda-Espinosa | > | Date | 06/3/2024 14:19 | > | To | | > | Subject | Re: Native Kubernetes Task Managers | > Ah no, I meant that I wouldn't use a statefu

Re: Native Kubernetes Task Managers

2024-06-02 Thread spoon_lz
` describes the pod index. Regards, Zhuo. Replied Message | From | Alexis Sarda-Espinosa | | Date | 06/3/2024 14:19 | | To | | | Subject | Re: Native Kubernetes Task Managers | Ah no, I meant that I wouldn't use a stateful set, rather just adjust the names of the pods that are created/ma

Re: Native Kubernetes Task Managers

2024-06-02 Thread Alexis Sarda-Espinosa
Ah no, I meant that I wouldn't use a stateful set, rather just adjust the names of the pods that are created/managed directly by the job manager. Regards, Alexis. Am Mo., 3. Juni 2024 um 07:31 Uhr schrieb Xintong Song < tonysong...@gmail.com>: > I may not have understood what you mean by the nam

Re: Native Kubernetes Task Managers

2024-06-02 Thread Xintong Song
I may not have understood what you mean by the naming scheme. I think the limitation "pods in a StatefulSet are always terminated in the reverse order as they are created" comes from Kubernetes and has nothing to do with the naming scheme. Best, Xintong On Mon, Jun 3, 2024 at 1:13 PM Alexis Sa

Re: Native Kubernetes Task Managers

2024-06-02 Thread Alexis Sarda-Espinosa
Hi Xintong, After experimenting a bit, I came to roughly the same conclusion: cleanup is what's more or less incompatible if Kubernetes manages the pods. Then it might be better to just allow using a more stable pod naming scheme that doesn't depend on the attempt number and thus produces more sta

Re: Native Kubernetes Task Managers

2024-06-02 Thread Xintong Song
I think the reason we didn't choose StatefulSet when introducing the Native K8s Deployment is that, IIRC, we want Flink's ResourceManager to have full control of the individual pod lifecycles. E.g., - Pods in a StatefulSet are always terminated in the reverse order as they are created. This preven