surahman edited a comment on issue #3724: URL: https://github.com/apache/incubator-heron/issues/3724#issuecomment-971908485
I have a plan in place to begin the first PR (Phase 1) once I know the PVC PR is stable. * Executor containers will retain the name `executor`. * Manager containers will have the name set to `manager`. * Commands to set CPU limit: `heron.kubernetes.manager.limit.cpu` and RAM limit: `heron.kubernetes.manager.limit.memory`. * `StatefulSet` for both will be duplicated. * `executor` container boot argument will have `shard_id` in the range `[1 - n)`. * `manager` container boot argument will have `shard_id` set to `0`. * Create a `Service` with a single open TCP/IP port to facilitate communication. `Selector` set to match `Labels` for `topology` and `app`. Generate a re-creatable name using the `topology` name. * `Selector` `matchLabels` for all toplogies `StatefulSet`s and `Service` set to `topology` and `app`. * Remove both `StatefulSet`s using the [`deleteCollectionNamespacedStatefulSet`](https://javadoc.io/static/io.kubernetes/client-java-api/11.0.0/io/kubernetes/client/openapi/apis/AppsV1Api.html#deleteCollectionNamespacedStatefulSet-java.lang.String-java.lang.String-java.lang.String-java.lang.String-java.lang.String-java.lang.Integer-java.lang.String-java.lang.Integer-java.lang.Boolean-java.lang.String-java.lang.String-java.lang.String-java.lang.Integer-io.kubernetes.client.openapi.models.V1DeleteOptions-) with the `topology` and `apps` `Labels`. * Remove the `Service` with [`deleteNamespacedService`](https://javadoc.io/static/io.kubernetes/client-java-api/11.0.0/io/kubernetes/client/openapi/apis/CoreV1Api.html#deleteNamespacedService-java.lang.String-java.lang.String-java.lang.String-java.lang.String-java.lang.Integer-java.lang.Boolean-java.lang.String-io.kubernetes.client.openapi.models.V1DeleteOptions-). I think that should cover everything required to perform a basic decoupling of the `manager` and `executor` pods. We should also visit whether each `executor` container needs to have debugging ports to the tune of the number of Pods in the topology. On the topic of creating a `Service`, I am wondering if we cannot just use the one which is already created for the topology: https://github.com/apache/incubator-heron/blob/6b883bec716df6c58b86a5e3f12970ab4e72c1c6/heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/V1Controller.java#L124-L131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
