surahman edited a comment on issue #3707:
URL:
https://github.com/apache/incubator-heron/issues/3707#issuecomment-911959524
A quick review of the code in the
`heron/schedulers/src/java/org/apache/heron/scheduler/kubernetes/` directory,
and some other related code:
* `KubernetesConstants`: namespaced constants for configurations.
* `KubernetesContext`: child class of `Context` with Kubernetes specific
keys used to lookup values (no "setters", only "getters") stored in the
supplied `Config` object.
* `KubernetesController`: the basic interface with Kubernetes to recover
info about the topology configs as well as submit/kill/restart them.
* `KubernetesLauncher`: used to submit a topology to the Kubernetes cluster.
* `KubernetesScheduler`: the interface to manage the cluster, get info,
kill, restart, resize, and update the cluster size based on the topology. It
contains a `KubernetesController` object which facilitates the comms with the
cluster.
* `KubernetesUtils`: various logging utilities.
* `V1Controller`: child class of the `KubernetesController` which contains
the actual logic to interface with the Kubernetes cluster. The `PackingPlan`
for the instances is central to operations with configurations stored in a
`ContainerPlan` object.
* `Volumes`: utilities used to configure the clusters storage volumes.
* `PackingPlan`: `heron/spi/src/java/org/apache/heron/spi/packing/` has the
classes for `PackingPlan`, `InstancePlan` and `ContainerPlan`.
I have very quickly compiled a basic `Pod Config`:
```YAML
# POD CONFIG WITH TEMPLATE:
apiVersion: v1
kind: Pod
metadata:
name: heron-node
spec:
replicas: 2
selector:
matchLabels:
app: heron-node
template:
metadata:
labels:
app: heron-node
spec:
containers:
- name: heron-node
image: centos:8
command: ['sh', '-c', 'echo "Heron Kubernetes node." && sleep 3600']
resources:
requests:
memory: "1Gb"
cpu: "250m"
limits:
memory: "5Gb"
cpu: "500m"
securityContext:
allowPrivilegeEscalation: false
capabilities:
add: ["NET_ADMIN", "SYS_TIME"]
restartPolicy: OnFailure
```
The code
[snippet](https://github.com/apache/spark/blob/de59e01aa4853ef951da080c0d1908d53d133ebe/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala#L39-L54)
from Spark is appending the following to the pod config, which creates a
`Volume` mount pointing to the pod config template:
```YAML
volumes:
- name: pod-template-name # from <POD_TEMPLATE_VOLUME>.
configMap:
name: configmap-name # from <configmapName>.
items:
- key: pod-template-key # from <POD_TEMPLATE_KEY>.
path: executor-pod-spec-template-file-name # from
<EXECUTOR_POD_SPEC_TEMPLATE_FILE_NAME>.
```
This code
[snippet](https://github.com/apache/spark/blob/de59e01aa4853ef951da080c0d1908d53d133ebe/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala#L56-L61)
is adding the `Volume` mount pointing to a container config template:
```YAML
volumes:
- name: pod-template-name # from <POD_TEMPLATE_VOLUME>.
path: executor-pod-spec-template-mount-path # from
<EXECUTOR_POD_SPEC_TEMPLATE_MOUNTPATH>.
```
The constants (caps naming convention) are defined
[here](https://github.com/apache/spark/blob/de59e01aa4853ef951da080c0d1908d53d133ebe/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Constants.scala#L81-L85).
If we are to follow what Spark is doing, we will need to declare these
constants as conventions and document them. The most prudent place for these
constants is `KubernetesConstants` as they will not be required outside. The
`configmapName` is being acquired as
[such](https://github.com/apache/spark/blob/de59e01aa4853ef951da080c0d1908d53d133ebe/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala#L37).
The remaining two routines in the Spark code appear to be extracting
information from the configs - they are not modifying them.
The question now is what functionality exists within Heron to modify the
`Config` object to append the YAML elements to the `Config` as well as where
(object) the actual pod/container configs are stored within the `Config`.
Another question is if there is support for both containers and pods configs,
and what the structure within the `Config` object is. Once we have an
understanding of these things I can hammer out a solution.
I have never used Heron and am learning about the workflow and setup as I
go, so please bear with me. I am also still getting familiar with the vast
codebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
