[
https://issues.apache.org/jira/browse/AIRFLOW-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740260#comment-16740260
]
Summit Suen commented on AIRFLOW-2955:
--------------------------------------
Hey I just ran into this issue as well and I would like to fix it, as a first
time contributor what should I be cautious about?
> Kubernetes pod operator: Unable to set requests/limits on task pods
> -------------------------------------------------------------------
>
> Key: AIRFLOW-2955
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2955
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Jon Davies
> Priority: Major
>
> When I try and set a resource limit/request on a DAG task with the
> KubernetesPodOperator as follows:
> {code:java}
> resources={"limit_cpu": 1, "request_cpu": 1},
> {code}
> ...I get:
> {code:java}
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task Traceback (most recent call last):
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/bin/airflow", line 32, in <module>
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task args.func(args)
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py",
> line 74, in wrapper
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task return f(*args, **kwargs)
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line
> 498, in run
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task _run(args, dag, ti)
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/lib/python3.7/site-packages/airflow/bin/cli.py", line
> 402, in _run
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task pool=args.pool,
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task return func(*args, **kwargs)
> [2018-08-24 15:51:27,795] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File "/usr/local/lib/python3.7/site-packages/airflow/models.py", line
> 1633, in _run_raw_task
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task result = task_copy.execute(context=context)
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/kubernetes_pod_operator.py",
> line 115, in execute
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task get_logs=self.get_logs)
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/kubernetes/pod_launcher.py",
> line 71, in run_pod
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task resp = self.run_pod_async(pod)
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/kubernetes/pod_launcher.py",
> line 52, in run_pod_async
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task req = self.kube_req_factory.create(pod)
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/kubernetes/kubernetes_request_factory/pod_request_factory.py",
> line 56, in create
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task self.extract_resources(pod, req)
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/kubernetes/kubernetes_request_factory/kubernetes_request_factory.py",
> line 160, in extract_resources
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task if not pod.resources or pod.resources.is_empty_resource_request():
> [2018-08-24 15:51:27,796] {base_task_runner.py:107} INFO - Job 2: Subtask
> task AttributeError: 'dict' object has no attribute
> 'is_empty_resource_request'
> {code}
> ...setting
> https://github.com/apache/incubator-airflow/blob/fc10f7e0a04145a0b2f31f8d0990bbe900b4e8a2/airflow/example_dags/example_kubernetes_executor.py#L66
> works, however that only adjusts the metadata for the worker pod and not the
> pod ultimately used for the task.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
