GitHub user pruyai created a discussion: Error observed: AirflowRuntimeError in
airflow version 3.1.2
Hello people , I am facing an issue in my airflow code. I am working with
celery executer and airflow version 3.1.2. It seems like when multiple tasks
are rendering fields in parallel, the insert conflicts and raises the error
above. I have once observed this happening when single task tries to execute as
well but usually seen with parallel executions.
AirflowRuntimeError: API_SERVER_ERROR: {'status_code': 409, 'message': 'Server
returned error', 'detail': {'detail': {'reason': 'Unique constraint violation',
'statement': 'INSERT INTO rendered_task_instance_fields (dag_id, task_id,
run_id, map_index, rendered_fields, k8s_pod_yaml) VALUES (%(dag_id)s,
%(task_id)s, %(run_id)s, %(map_index)s, %(rendered_fields)s,
%(k8s_pod_yaml)s)', 'orig_error': 'duplicate key value violates unique
constraint "rendered_task_instance_fields_pkey"
This issue seems to be intermittent and happens with multiple tasks, sometimes
even 3 attempts for task resolution results into the same error causing entire
dag failure.
Is this a known issue and would be resolved sooner in next couple of releases?
Could you please guide if there is any way to fix this up. Any suggestions
would be highly appreciated.
GitHub link: https://github.com/apache/airflow/discussions/61442
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
