Hi,
I am working on [YUNIKORN-86] (Set proper resource request and limit for
YuniKorn pods) issues and I would like to hear your feedback on this issue.
Link:https://issues.apache.org/jira/browse/YUNIKORN-86
Currently I am working on finding a justified request and limit value for
the scheduler and shim pods in k8s. So far:
- Researched a bit on the k8s default scheduler (kube-scheduler) and even
though it can be configured, that are no defaults for that pod. On AKS
there is a default that I could find on the internet:
>>>
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 4
memory: 2Gi
>>>
- Tried to obtain some values from a deployed k8s cluster, but had troubles
with the metrics-server (in some old version there was heapster deployed)
- Ran the yunikorn-core/pkg/scheduler/tests/scheduler_perf_tests.go and
monitored the cpu/memory consumption. I don't have the run's full graph,
but these are the approximate values that I experienced
- for most of the time cpu was on around 4 cores
- on peak time it was around 5.2 cores
- the memory consumption was moderate, it was around 4.3% of the 16Gb
memory of my laptop - which is around 700Mb when the perf test was ended.
The memory depends on the stored/currently running applications, so I think
1Gb should be fine for this for general purpose.
Note that the cpu numbers are related to the peak usage, when the scheduler
was under pressure, but when only a small amount of pods are
scheduled, this is a much smaller number.
I suggest the following (similar to AKS) number:
>>>
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 4
memory: 2Gi
>>>
One additional thing: there were some measurements of YuniKorn in kubemark.
Could you please share your outputs if there's something related to the
scheduler and the shim pod's resource usage?
Regards,
Adam
