xintongsong commented on a change in pull request #14629: URL: https://github.com/apache/flink/pull/14629#discussion_r580876622
########## File path: docs/content/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,235 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager pods via template files, with the advanced features (e.g. volumes, init containers, sidecar containers, etc.) +that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used, instead of an empty pod, to initialize the +JobManager and TaskManager. The main container should be defined with name "flink-main-container". Review comment: ```suggestion JobManager and TaskManager. The main container should be defined with name `flink-main-container`. ``` ########## File path: docs/content/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,235 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager pods via template files, with the advanced features (e.g. volumes, init containers, sidecar containers, etc.) +that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used, instead of an empty pod, to initialize the +JobManager and TaskManager. The main container should be defined with name "flink-main-container". +Please refer to the [pod template example](#example-of-pod-template) for more information. + +#### Fields Overwritten by Flink + +Some fields of the pod template can be overwritten by Flink. +The overwritten mechanism can be categorized as follows. +* **Defined by Flink.** Definitions in the template won't take effect. User cannot control the fields. +* **Defined by Config Option.** Definitions in the template won't take effect. User can control the fields via config options. +* **Overwritten by Flink.** Definitions in the template will take effect, with additional contents from Flink. + +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template that are not listed in the tables will be unaffected. + +<span class="label label-info">Note</span> Flink may overwrite more fields from pod templates in future releases. +While it is undetermined which fields might be overwritten in the future, Flink provides the following list of fields that are planned to be long-term supported: `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, `env`, `init container`, `sidecar container`, `volumes`. + +##### Pod Metadata +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 10%">Key</th> + <th class="text-left" style="width: 20%">Category</th> + <th class="text-left" style="width: 30%">Related Config Options</th> + <th class="text-left" style="width: 40%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>name</td> + <td>Defined by Flink</td> + <td></td> + <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>. + The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td> + </tr> + <tr> + <td>namespace</td> + <td>Defined by Config Option</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td> + <td>Both the JobManager deployment and TaskManager pods will be created in the specified namespace.</td> + </tr> + <tr> + <td>ownerReferences</td> + <td>Defined by Config Option</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a></td> + <td>The owner reference of JobManager deployment could be set by configuration option. + And the owner reference of every TaskManager pod will be set to JobManager deployment.</td> + </tr> + <tr> + <td>annotations</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td> + <td>Flink will add additional annotations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>labels</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td> + <td>Flink will add additional labels specified by the Flink configuration options.</td> + </tr> + </tbody> +</table> + +##### Pod Spec +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 10%">Key</th> + <th class="text-left" style="width: 20%">Category</th> + <th class="text-left" style="width: 30%">Related Config Options</th> + <th class="text-left" style="width: 40%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>imagePullSecrets</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-secrets">kubernetes.container.image.pull-secrets</a></td> + <td>Flink will add additional pull secrets specified by the Flink configuration options.</td> + </tr> + <tr> + <td>nodeSelector</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-node-selector">kubernetes.jobmanager.node-selector</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-node-selector">kubernetes.taskmanager.node-selector</a></td> + <td>Flink will add additional node selectors specified by the Flink configuration options.</td> + </tr> + <tr> + <td>tolerations</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-tolerations">kubernetes.jobmanager.tolerations</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-tolerations">kubernetes.taskmanager.tolerations</a></td> + <td>Flink will add additional tolerations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>restartPolicy</td> + <td>Defined by Flink</td> + <td></td> + <td>"always" for JobManager pod and "never" for TaskManager pod. + <br> + The JobManager pod will always be restarted by deployment. And the TaskManager pod should not be restarted.</td> + </tr> + <tr> + <td>serviceAccount</td> + <td>Defined by Config Option</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-service-account">kubernetes.service-account</a></td> + <td>The JobManager deployment and TaskManager pods will be created with the specified service account.</td> + </tr> + <tr> + <td>volumes</td> + <td>Overwritten by Flink</td> + <td></td> + <td>Flink will add some additional internal ConfigMap volumes(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td> + </tr> + </tbody> +</table> + +##### Main Container Spec Review comment: ```suggestion **Main Container Spec** ``` ########## File path: docs/content/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,235 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager pods via template files, with the advanced features (e.g. volumes, init containers, sidecar containers, etc.) +that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used, instead of an empty pod, to initialize the +JobManager and TaskManager. The main container should be defined with name "flink-main-container". +Please refer to the [pod template example](#example-of-pod-template) for more information. + +#### Fields Overwritten by Flink + +Some fields of the pod template can be overwritten by Flink. +The overwritten mechanism can be categorized as follows. +* **Defined by Flink.** Definitions in the template won't take effect. User cannot control the fields. +* **Defined by Config Option.** Definitions in the template won't take effect. User can control the fields via config options. +* **Overwritten by Flink.** Definitions in the template will take effect, with additional contents from Flink. + +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template that are not listed in the tables will be unaffected. + +<span class="label label-info">Note</span> Flink may overwrite more fields from pod templates in future releases. +While it is undetermined which fields might be overwritten in the future, Flink provides the following list of fields that are planned to be long-term supported: `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, `env`, `init container`, `sidecar container`, `volumes`. + +##### Pod Metadata Review comment: ```suggestion **Pod Metadata** ``` ########## File path: docs/content/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,235 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager pods via template files, with the advanced features (e.g. volumes, init containers, sidecar containers, etc.) +that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used, instead of an empty pod, to initialize the +JobManager and TaskManager. The main container should be defined with name "flink-main-container". +Please refer to the [pod template example](#example-of-pod-template) for more information. + +#### Fields Overwritten by Flink + +Some fields of the pod template can be overwritten by Flink. +The overwritten mechanism can be categorized as follows. +* **Defined by Flink.** Definitions in the template won't take effect. User cannot control the fields. +* **Defined by Config Option.** Definitions in the template won't take effect. User can control the fields via config options. +* **Overwritten by Flink.** Definitions in the template will take effect, with additional contents from Flink. + +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template that are not listed in the tables will be unaffected. + +<span class="label label-info">Note</span> Flink may overwrite more fields from pod templates in future releases. +While it is undetermined which fields might be overwritten in the future, Flink provides the following list of fields that are planned to be long-term supported: `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, `env`, `init container`, `sidecar container`, `volumes`. + +##### Pod Metadata +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 10%">Key</th> + <th class="text-left" style="width: 20%">Category</th> + <th class="text-left" style="width: 30%">Related Config Options</th> + <th class="text-left" style="width: 40%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>name</td> + <td>Defined by Flink</td> + <td></td> + <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>. + The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td> + </tr> + <tr> + <td>namespace</td> + <td>Defined by Config Option</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td> + <td>Both the JobManager deployment and TaskManager pods will be created in the specified namespace.</td> + </tr> + <tr> + <td>ownerReferences</td> + <td>Defined by Config Option</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a></td> + <td>The owner reference of JobManager deployment could be set by configuration option. + And the owner reference of every TaskManager pod will be set to JobManager deployment.</td> + </tr> + <tr> + <td>annotations</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td> + <td>Flink will add additional annotations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>labels</td> + <td>Overwritten by Flink</td> + <td><a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a> + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td> + <td>Flink will add additional labels specified by the Flink configuration options.</td> + </tr> + </tbody> +</table> + +##### Pod Spec Review comment: ```suggestion **Pod Spec** ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
