afusr created AIRFLOW-3819:
------------------------------
Summary: k8s executor - Allow the configuration of a global
default for pod resource request/limits
Key: AIRFLOW-3819
URL: https://issues.apache.org/jira/browse/AIRFLOW-3819
Project: Apache Airflow
Issue Type: Improvement
Components: kubernetes
Affects Versions: 1.10.2
Reporter: afusr
Currently the kubernetes executor allows you to specify pod resources requests
and limits (cpu and memory). For example:
{noformat}
# Limit resources on this operator/task with node affinity & tolerations
three_task = PythonOperator(
task_id="three_task", python_callable=print_stuff, dag=dag,
executor_config={
"KubernetesExecutor": {"request_memory": "128Mi",
"limit_memory": "128Mi",
"tolerations": tolerations,
"affinity": affinity}}
)
{noformat}
These values are used by kubernetes when making scaling decisions. It would be
nice to be able to specify a global default for these values, to ensure that
each pod airflow creates has a value specified for these properties.
There is still the requirement to override these values on a dag by dag basis.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
