This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 1baa7520bea Add basic Airflow error guide (#44616)
1baa7520bea is described below
commit 1baa7520bea853b199430543fe85f0c352f9dd13
Author: Omkar P <[email protected]>
AuthorDate: Sat Feb 8 21:57:18 2025 +0000
Add basic Airflow error guide (#44616)
* Add basic Airflow error guide
* Run static checks
* Update AERR markdown list with top 100 user facing errors
* Re-run static checks
* Add first steps and documentation link to guide
---
dev/AIRFLOW_ERROR_GUIDE.md | 131 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 131 insertions(+)
diff --git a/dev/AIRFLOW_ERROR_GUIDE.md b/dev/AIRFLOW_ERROR_GUIDE.md
new file mode 100644
index 00000000000..da45aa8df1a
--- /dev/null
+++ b/dev/AIRFLOW_ERROR_GUIDE.md
@@ -0,0 +1,131 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update
-->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents** *generated with
[DocToc](https://github.com/thlorenz/doctoc)*
+
+- [Airflow Error Guide](#airflow-error-guide)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+# Airflow Error Guide
+
+|Error Code| Exception Type | User-facing
Error Message |
Description |
First Steps
|
Documentation [...]
+|----------|----------------------------|---------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------
[...]
+| AERR001 | AirflowException | Dynamic task
mapping exceeded limit | Happens when
dynamically mapped tasks exceed the maximum number of tasks allowed.
| Check the task count limit in the
configuration. Consider increasing the task limit or optimizing task mapping
logic. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dynami [...]
+| AERR002 | AirflowException | Task
instance not found | Happens when the
scheduler or webserver cannot locate a task instance in the database.
| Verify that the database connection is stable. Check if
the task instances exist in the metadata database and consider re-running the
DAG. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/ [...]
+| AERR003 | AirflowException | Task is in
'None' state | Indicates a task instance has not
been assigned a proper state, often due to missing execution context. |
Ensure that the task is correctly configured and the
execution context is provided, especially for dynamic tasks or task
dependencies. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag-run. [...]
+| AERR004 | AirflowWebServerException | Webserver
502 Bad Gateway | Triggered when the
webserver encounters an upstream issue or fails to proxy requests. |
Check the webserver logs for more details. Ensure all
upstream systems are working properly, and restart the webserver if necessary.
|
https://airflow.apache.org/docs/apache-airflow/stable/webserver.htm [...]
+| AERR005 | KeyError | KeyError in
Variable retrieval | Occurs when a requested
Airflow Variable is not found in the metadata database. |
Check if the Airflow variable exists. Ensure the correct database is used,
and verify if the Airflow variable is defined in the Airflow UI or through
code. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/ [...]
+| AERR006 | PermissionError | Access
denied for SSH hook | Triggered when the
SSH hook cannot authenticate or connect to the target server. |
Verify the SSH credentials and network access to the target
server. Test the connection using a manual SSH client before troubleshooting in
Airflow. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/ope [...]
+| AERR007 | AirflowXComException | TaskInstance not
recognized in XCom | Happens when a task's XCom
entry is missing or corrupted in the metadata database. |
Check if the XCom data is being pushed correctly. Inspect the DAG code for
issues in data push logic, and consider clearing any corrupted XCom entries.
|
https://airflow.apache.org/docs/apache-airflow/stable/concept [...]
+| AERR008 | AirflowDatabaseException | Duplicate XCom
entry detected | Occurs when the same XCom
key-value pair is inserted multiple times into the database. |
Ensure that the XCom key-value pair is unique for each task.
Modify the DAG logic to avoid reusing keys or overwriting XCom entries.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html [...]
+| AERR009 | AirflowDatabaseException | Error creating
database session | Triggered when
Airflow cannot create a new database session. |
Check the database connection settings and ensure the database is
running. Verify user permissions and the number of concurrent connections
allowed. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/set-u [...]
+| AERR010 | AirflowConfigException | Strict validation in
Dataset URI breaks existing DAGs | Happens when a dataset URI is
not compliant with stricter validation rules introduced in newer versions. |
Review the dataset URI format for compliance with new
validation rules. Update the URI if necessary to meet the required standards.
|
https://airflow.apache.org/docs/apache-airflow/stable/datasets.htm [...]
+| AERR011 | AirflowException | Failed to upload
logs to remote storage | Occurs when Airflow
cannot push task logs to a configured remote storage backend. |
Check the configuration for the remote storage backend.
Ensure that the connection credentials are correct and the backend is
accessible. |
https://airflow.apache.org/docs/apache-airflow/stable/logging.htm [...]
+| AERR012 | AirflowDatabaseException | Cannot connect
to airflow database | Happens when the metadata
database is unreachable due to network or configuration issues. |
Verify the network connection to the metadata database and
check for any misconfigurations. Restart Airflow components if necessary.
|
https://airflow.apache.org/docs/apache-airflow/stable/administration/m [...]
+| AERR013 | KeyError | KeyError in
retrieving XCom value | Occurs when a
requested XCom value is not found or incorrectly defined. |
Ensure the XCom key and value are properly defined and
passed between tasks. Double-check task execution order and parameter passing.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html# [...]
+| AERR014 | ImportError | Missing dependency
for KubernetesExecutor | Occurs when the required
dependencies for the KubernetesExecutor are not installed. |
Verify that all required dependencies for the KubernetesExecutor are
installed. Use `pip` or your environment management tool to install missing
packages. |
https://airflow.apache.org/docs/apache-airflow/stable/executors/kubernetes [...]
+| AERR015 | AirflowDagPausedException | DAG is paused
and not running | Indicates the DAG is
manually paused and will not trigger scheduled runs. |
Check the DAG status in the Airflow UI and unpause the
DAG if needed. Verify the DAG configuration and dependencies.
|
https://airflow.apache.org/docs/apache-airflow/stable/dag-run.ht [...]
+| AERR016 | AirflowTaskTimeout | Task execution
delayed indefinitely | Happens when a task
does not start execution within the specified timeout. |
Review the task timeout settings in the DAG configuration.
Increase the timeout if necessary and check for system performance issues.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR017 | AirflowConfigException | Can't find
executor class | Happens when the executor
specified in the configuration file is not recognized or available. |
Verify the executor configuration in the `airflow.cfg` file.
Ensure the specified executor is installed and supported by your Airflow
version. |
https://airflow.apache.org/docs/apache-airflow/stable/executor [...]
+| AERR018 | AirflowConfigException | Invalid value in
airflow.cfg file | Triggered when
airflow.cfg contains an invalid or unsupported value. |
Review the `airflow.cfg` configuration file for errors or
unsupported values. Consult the Airflow documentation for valid settings.
|
https://airflow.apache.org/docs/apache-airflow/stable/configura [...]
+| AERR019 | AirflowCliException | Airflow CLI
authentication failed | Occurs when CLI
commands cannot authenticate with the Airflow backend. |
Check if the Airflow CLI has the proper credentials to access the
backend. Ensure the correct connection configurations and environment variables
are set. |
https://airflow.apache.org/docs/apache-airflow/stable/cli-and-commands [...]
+| AERR020 | AirflowTriggerException | Error
triggering external API | Happens
when a trigger for an external API fails to execute. |
Check the API endpoint and ensure it is reachable.
Review the Airflow logs to see if there is an error in the API trigger logic.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator [...]
+| AERR021 | AirflowDatabaseException | Database
deadlock detected | Occurs when multiple
processes are locked in conflicting database operations. |
Investigate the database for any deadlocks or conflicting
operations. Optimize queries or increase database capacity to avoid locking
issues. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/data [...]
+| AERR022 | PermissionError | Permission error in
KubernetesPodOperator | Occurs when the
KubernetesPodOperator lacks permissions to perform required actions.
| Review the KubernetesPodOperator configuration and
ensure it has the necessary permissions to perform actions in your Kubernetes
cluster. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/kube [...]
+| AERR023 | AirflowSchedulerException | Scheduler
loop error | Triggered when the
scheduler encounters an unexpected condition during its loop. |
Check the scheduler logs for specific error messages. Review
recent changes to the Airflow environment or DAGs that could affect scheduler
behavior. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/scheduler.
[...]
+| AERR024 | AirflowParseException | Broken DAG:
syntax error | Occurs when a syntax
error in the DAG file prevents it from being parsed. |
Review the DAG file for any syntax errors and correct
them. Use a linter or Python syntax checker to help identify issues.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/ [...]
+| AERR025 | AirflowDatabaseException | DagRun state
update failed | Occurs when Airflow
fails to update the state of a DAG run in the database. |
Check the database connection and permissions. Review any
database constraints or performance issues that could prevent state updates.
|
https://airflow.apache.org/docs/apache-airflow/stable/administration/m [...]
+| AERR026 | AirflowTaskTimeout | Task marked as
failed due to timeout | Happens when a
task exceeds its maximum allowable execution time. |
Increase the task execution timeout in the DAG configuration.
Investigate the task logic for inefficiencies or external system delays.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR027 | FileNotFoundError | Task log
not found | Happens when task
logs are missing from the local or remote storage. |
Check the task log configuration in Airflow. Ensure that logging paths
are correctly configured and accessible. Verify permissions for remote storage.
|
https://airflow.apache.org/docs/apache-airflow/stable/logging.ht [...]
+| AERR028 | ImportError | Cannot import
module in BashOperator | Triggered when a
script run by BashOperator references missing Python modules. |
Ensure all required Python modules are installed in the
environment. Check the `requirements.txt` or virtual environment for missing
dependencies. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/b [...]
+| AERR029 | AirflowConfigException | Error loading
connections from secret | Happens when Airflow
fails to load connection credentials from a secret backend. |
Check the secret backend configuration and credentials.
Ensure the secret backend is accessible and correctly set up in Airflow.
|
https://airflow.apache.org/docs/apache-airflow/stable/security [...]
+| AERR030 | AirflowWorkerException | Worker not
responding | Happens when a worker
node becomes unresponsive or fails to report its status. |
Check the worker node logs for errors. Restart the worker node if
necessary and verify network connectivity between the scheduler and worker.
|
https://airflow.apache.org/docs/apache-airflow/stable/administrat [...]
+| AERR031 | AirflowException | Resource not
found in GCP hook | Triggered when a
GCP hook is unable to locate the specified resource. |
Verify the resource exists in GCP. Check the GCP credentials
and ensure that the hook is correctly configured to access the resource.
|
https://airflow.apache.org/docs/apache-airflow-providers-google/s [...]
+| AERR032 | AirflowExecutorException | Backend not
reachable for Celery | Occurs when the
CeleryExecutor cannot connect to the configured Celery backend. |
Check the Celery backend configuration in `airflow.cfg`.
Verify network access and that the Celery worker is properly connected and
running. |
https://airflow.apache.org/docs/apache-airflow/stable/executor [...]
+| AERR033 | AirflowException | Invalid
cron expression | Occurs when the cron
schedule provided in the DAG is invalid or unparsable. |
Check the cron expression syntax for errors. Use a cron
expression validator tool to confirm it is valid and properly formatted.
|
https://airflow.apache.org/docs/apache-airflow/stable/scheduler.htm [...]
+| AERR034 | UnpicklingError | UnpicklingError
while running task | Happens when Airflow cannot
deserialize data, often due to incompatible Python versions or corrupted data.
| Ensure that Airflow and its dependencies are compatible
with the Python version in use. Review any corrupted data and clear invalid
entries. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/se [...]
+| AERR035 | AirflowWorkerException | Task instance
killed by external system | Occurs when an
external system terminates a task instance during execution. |
Review the external system logs to determine why it
terminated the task. Modify task handling to account for external termination
events. |
https://airflow.apache.org/docs/apache-airflow/stable/operators.html# [...]
+| AERR036 | AirflowWorkerException | Worker died
during task execution | Happens when the
worker process handling a task crashes or is terminated. |
Investigate the worker logs to identify the cause of the crash.
Ensure that the worker environment has adequate resources and is properly
configured. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/work [...]
+| AERR037 | AirflowDatabaseException | Failed to
fetch task state | Occurs when the metadata
database does not return a valid state for a task instance. |
Check the metadata database for data consistency and ensure that the
database is responsive. Investigate for any database corruption or
misconfigurations. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/m [...]
+| AERR038 | ValueError | Cron interval
parsing failed | Occurs when the cron
expression for a DAG's schedule interval cannot be parsed. |
Review the cron expression in the DAG configuration. Use a cron
expression validator to confirm the format is correct and supported by Airflow.
|
https://airflow.apache.org/docs/apache-airflow/stable/scheduler.htm [...]
+| AERR039 | AirflowSchedulerException | Scheduler throttled
due to excessive DAGs | Happens when the
scheduler takes too long to process a large number of DAGs. |
Consider optimizing DAG execution and task scheduling. Check the scheduler logs
for any performance bottlenecks, and adjust the scheduler settings to handle
large DAGs efficiently. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/schedule
[...]
+| AERR040 | AirflowDatabaseException | DagRun
execution_date conflicts | Occurs when there
is a mismatch in execution_date for a DAG run in the database. |
Ensure the execution_date is correctly defined and consistent
across task instances. Check for any issues with time zone settings or manual
overrides. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts [...]
+| AERR041 | AirflowTaskTimeout | Task is stuck
in queued state | Occurs when a task
remains queued without being picked up by an executor. |
Check the executor configuration and ensure that sufficient worker nodes are
available. Verify the queue settings and make sure the task is being routed to
the correct executor. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/tas [...]
+| AERR042 | AirflowParseException | Error while
parsing DAG file | Triggered when the
scheduler encounters syntax errors or invalid code in a DAG file. |
Review the DAG file for any code errors or invalid
syntax. Use a Python linter to catch issues before loading the DAG into
Airflow. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/ [...]
+| AERR043 | AirflowDagCycleException | Task
dependency cycle detected | Triggered
when task dependencies in a DAG create an infinite loop. |
Check task dependencies in the DAG and ensure that
there are no circular references. Modify dependencies to prevent infinite
loops. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.ht [...]
+| AERR044 | AirflowTaskException | Task failed due
to retries exceeded | Triggered when a
task exhausts its retry limit without succeeding. |
Increase the retry limit in the DAG configuration or
modify the task logic to handle failure scenarios more gracefully.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR045 | ValueError | ValueError in
task arguments | Happens when a task
operator is provided with invalid or incompatible arguments. |
Review the operator arguments and ensure they are correctly
specified. Check the Airflow documentation for valid arguments for the operator
used. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/o [...]
+| AERR046 | AirflowTaskException | Task
queue not found | Occurs when
the task's queue is not recognized by the executor. |
Ensure the specified queue exists in the Airflow configuration.
Review the executor settings to confirm it can process tasks from the specified
queue. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts [...]
+| AERR047 | AirflowSchedulerException | Executor cannot
retrieve task instance | Happens when the
executor fails to fetch task instance details from the database. |
Verify the database connection and ensure the task instance
details are present in the metadata database. Check the executor logs for any
errors. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/tas [...]
+| AERR048 | AirflowWebServerException | Webserver
connection refused | Occurs when
the webserver process is not running or accessible. |
Check the webserver logs for errors. Restart the webserver
and verify that it is running and accessible through the configured URL.
|
https://airflow.apache.org/docs/apache-airflow/stable/webse [...]
+| AERR049 | AirflowTaskException | Invalid return type
from PythonOperator | Happens when a
PythonOperator returns a value of an unexpected type. |
Ensure the Python function used in the PythonOperator returns
the expected type. Review the function implementation for type correctness.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/opera [...]
+| AERR050 | AirflowConfigException | Config error:
executor not defined | Triggered when the
Airflow configuration does not specify a valid executor. |
Review the `airflow.cfg` file and verify that the executor is correctly
specified. Ensure that the specified executor is installed and properly
configured. |
https://airflow.apache.org/docs/apache-airflow/stable/executor [...]
+| AERR051 | AirflowException | Error in task
failure hook execution | Triggered when the
failure hook defined for a task encounters an error. |
Check the failure hook configuration for errors. Review the task
and hook logs to identify the cause of the failure and resolve any issues.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator [...]
+| AERR052 | AirflowTemplateException | Failed to resolve
template variable | Triggered when a task's
templated field contains errors or undefined variables. |
Review the task templated fields for any errors or undefined
variables. Ensure all variables are defined and passed correctly in the DAG.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/operat [...]
+| AERR053 | AirflowSchedulerException | Scheduler process
killed unexpectedly | Triggered when the
scheduler process is terminated due to resource or system issues. |
Investigate the system logs for resource-related issues.
Increase system resources or adjust scheduler configurations to prevent
termination. |
https://airflow.apache.org/docs/apache-airflow/stable/administrati [...]
+| AERR054 | AirflowTaskTimeout | Task failed
with exit code 1 | Occurs when a task script
or subprocess exits with a non-zero code indicating failure. |
Check the task script or subprocess logs for the error code
and cause of failure. Debug the script and address any underlying issues.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/oper [...]
+| AERR055 | AirflowDatabaseException | TaskInstance
already exists in database | Occurs when a
duplicate TaskInstance entry is created in the metadata database.
| Review task instance scheduling and ensure that duplicate
entries are not being created. Investigate DAG scheduling logic for potential
issues. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/m [...]
+| AERR056 | AirflowDatabaseException | Could not
reach the database | Indicates connectivity
issues with the metadata database, often due to misconfiguration. |
Verify the connection settings in `airflow.cfg` for the metadata
database. Ensure that the database is accessible and network connectivity is
working. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-
[...]
+| AERR057 | AirflowExecutorException | Task stuck in
'deferred' state | Triggered when a task using
deferrable operators remains in 'deferred' longer than expected. |
Review the deferrable operator configuration and execution logic. Ensure
that the task is properly resumed and that there are no long delays in task
deferral. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operato [...]
+| AERR058 | AirflowException | Error loading
custom operator | Indicates a problem in
importing or defining a custom operator in the DAG. |
Check the custom operator import path and ensure it is available.
Verify that the custom operator class is correctly defined and instantiated.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator.html# [...]
+| AERR059 | AirflowTriggerException | Trigger timeout
for external task | Occurs when an
ExternalTaskSensor exceeds its timeout waiting for an external task.
| Increase the timeout setting for the
ExternalTaskSensor. Check if the external task is delayed and adjust the DAG
dependencies accordingly. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator [...]
+| AERR060 | AirflowDagImportException | DagBag
import errors | Happens when the
scheduler encounters issues loading DAG files into the DagBag. |
Check the DAG files for syntax or configuration errors.
Inspect the DagBag loading process to ensure no invalid DAGs are being
included. |
https://airflow.apache.org/docs/apache-airflow/stable/schedule [...]
+| AERR061 | AirflowDagNotFound | DAG not found
in trigger DAG run | Triggered when the DAG
specified in a TriggerDagRunOperator does not exist. |
Check the name of the target DAG in the TriggerDagRunOperator.
Ensure that the DAG exists and is correctly spelled in the configuration.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/operato [...]
+| AERR062 | AirflowDagCycleException | Circular
dependencies in DAG | Triggered
when tasks in a DAG form a circular dependency loop. |
Review the task dependencies and remove any circular
references. Modify the DAG structure to avoid infinite dependency loops.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.ht [...]
+| AERR063 | AirflowException | Error in
on_failure_callback for DAG | Occurs
when the on_failure_callback for a DAG raises an exception.
| Check the on_failure_callback function for errors. Ensure it
handles exceptions correctly and does not raise additional errors during task
failure. |
https://airflow.apache.org/docs/apache-airflow/stable/advanced [...]
+| AERR064 | AirflowTriggerException |
TriggerDagRunOperator failed | Happens when
the TriggerDagRunOperator cannot trigger the specified target DAG.
| Verify the configuration of the
TriggerDagRunOperator. Ensure the target DAG exists and the trigger parameters
are correct. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operato [...]
+| AERR065 | AirflowTaskException | DAG
execution failed | Indicates that the overall
execution of a DAG run failed due to errors in one or more tasks. |
Check the logs for each task to determine which
task failed. Review task dependencies, configurations, and retries.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag [...]
+| AERR066 | AirflowSchedulerException | Scheduler
heartbeat failed | Occurs when the scheduler
process stops sending heartbeats, often due to resource issues. |
Investigate the system resources, such as CPU and memory, and
increase them if necessary. Restart the scheduler and monitor system
performance. |
https://airflow.apache.org/docs/apache-airflow/stable/administrati [...]
+| AERR067 | AirflowConfigException | Scheduler crashes when passing
invalid value to argument in default_args | Occurs when
default_args in a DAG contains an invalid or incompatible parameter.
| Review the `default_args` section in the DAG
definition. Ensure all parameters are valid and compatible with the Airflow
version in use. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags [...]
+| AERR068 | AirflowApiException | HTTP error while
connecting to API | Occurs when Airflow encounters
connectivity issues or invalid responses from an external API. |
Check the API connection settings and ensure that the external
service is reachable. Verify that the API is returning valid responses.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/oper [...]
+| AERR069 | AirflowSchedulerException | Scheduler backlog
due to excessive retries | Happens when too
many tasks are retried, overloading the scheduler. |
Adjust the task retry logic and retry limits. Review task
logs for patterns that lead to excessive retries and optimize DAG logic.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR070 | AirflowException | DAG
folder not found | Triggered when the
directory specified for DAGs does not exist or is inaccessible. |
Ensure that the DAG directory exists and is accessible by
the Airflow scheduler and webserver. Check directory permissions and paths.
|
https://airflow.apache.org/docs/apache-airflow/stable/administration/ [...]
+| AERR071 | AirflowException | Max active tasks
for DAG exceeded | Triggered when the number of
concurrent tasks for a DAG exceeds the configured limit. |
Increase the task concurrency settings in the DAG configuration or
adjust the task dependencies to limit the number of concurrent tasks.
|
https://airflow.apache.org/docs/apache-airflow/stable/scheduler.h [...]
+| AERR072 | AirflowException | Cannot find DAG
run in the database | Happens when the specified
DAG run is missing or deleted from the metadata database. |
Verify the DAG run's existence in the metadata
database. Ensure the run has not been manually deleted or corrupted.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts [...]
+| AERR073 | AirflowException | Airflow CLI
not recognized | Happens when the Airflow
CLI command is not installed or available in the PATH. |
Ensure that Airflow is properly installed and that the CLI is
accessible in the environment PATH. Reinstall or reconfigure Airflow if
necessary. |
https://airflow.apache.org/docs/apache-airflow/stable/cli-and- [...]
+| AERR074 | AirflowDatabaseException | SQLAlchemy
database connection error | Happens when the
connection string for the metadata database is invalid or unreachable.
| Check the connection string in the Airflow configuration.
Ensure the database is reachable and verify the credentials and network
configuration. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/ [...]
+| AERR075 | PermissionError | Permission
denied for Airflow logs | Occurs when Airflow
does not have the required permissions to read or write logs. |
Check the file permissions for the Airflow logs
directory. Ensure that the user running Airflow has proper read and write
access. |
https://airflow.apache.org/docs/apache-airflow/stable/logg [...]
+| AERR076 | AirflowException | Task marked as 'up for
retry' without retries available | Happens when a task
is incorrectly marked for retry but has exceeded its limit. |
Check the task's retry configuration in the DAG. Ensure that the
retry count is properly set in the task's arguments and verify if retries are
needed. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR077 | AirflowConfigException | Invalid
section in airflow.cfg | Triggered when
airflow.cfg contains an unrecognized configuration section. |
Review your airflow.cfg file to ensure that all sections are correctly
defined and recognized. Consult the official Airflow configuration reference to
verify your settings. |
https://airflow.apache.org/docs/apache-airflow/stable/configura [...]
+| AERR078 | AirflowException | Task
dependencies are not met | Indicates a task
cannot start due to upstream dependencies not being completed. |
Ensure that all upstream tasks have been successfully completed before the
task is triggered. Review the task dependencies in your DAG to make sure they
are properly defined. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/ta [...]
+| AERR079 | AirflowException | Task not found
in serialized DAG | Occurs when the task
referenced is missing from the serialized DAG in the database. |
Check your DAG serialization settings and verify that all tasks
are properly serialized. Ensure that the task is defined correctly in the DAG
file. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.ht [...]
+| AERR080 | AirflowException | Worker node
not available | Triggered when an
executor cannot find a suitable worker to run a task. |
Check if there are any worker nodes available and verify if the executor
configuration is correct. Ensure the workers are running and properly
configured in the Airflow setup. |
https://airflow.apache.org/docs/apache-airflow/stable/executor [...]
+| AERR081 | ModuleNotFoundError | Python
dependency not installed | Occurs
when a Python package required by a DAG is missing. |
Verify that all required Python packages are installed in the environment where
Airflow is running. Install any missing dependencies using `pip` or a virtual
environment manager. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/pyth [...]
+| AERR082 | AirflowExecutorException | Celery task
not acknowledged | Occurs when a
Celery task is not acknowledged by a worker. |
Check the Celery worker logs to determine why the task was not
acknowledged. Ensure that Celery is properly configured and running in your
Airflow setup. |
https://airflow.apache.org/docs/apache-airflow/stable/executor [...]
+| AERR083 | AirflowTaskTimeout | Task duration
exceeds timeout | Happens when a
task exceeds its specified execution timeout. |
Review the task's execution timeout setting. If necessary, increase the
timeout or optimize the task to ensure it completes within the allowed
duration. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR084 | AirflowException | Error migrating database
to Airflow version 2.9.3 | Indicates issues during
metadata database migration, often due to schema mismatches. |
Check the database schema and migration logs for errors. You may need to run
manual database migrations or troubleshoot database connectivity and
compatibility issues. |
https://airflow.apache.org/docs/apache-airflow/stable/administration/da [...]
+| AERR085 | AirflowException | Error writing
metrics to statsd | Occurs when Airflow
fails to send metrics data to the configured statsd server. |
Verify the connection settings to your statsd server. Ensure
that the statsd server is reachable and properly configured in Airflow's
settings. |
https://airflow.apache.org/docs/apache-airflow/stable/metr [...]
+| AERR086 | AirflowException | DAG
import timeout | Occurs when DAG files take
too long to parse, possibly due to large files or inefficient code. |
Review your DAG files to identify any large or inefficient code. Optimize
DAG parsing and consider splitting large DAGs into smaller ones to improve
performance. |
https://airflow.apache.org/docs/apache-airflow/stable/scheduler.h [...]
+| AERR087 | AirflowTemplateException | Invalid template in
BashOperator command | Happens when the
command template in BashOperator contains errors. |
Check the BashOperator template for syntax errors. Ensure all placeholders
and variables are correctly defined and are available in the context of the
task. |
https://airflow.apache.org/docs/apache-airflow/stable/howto/oper [...]
+| AERR088 | AirflowDagCycleException | Multiple
DAGs with same ID | Triggered when
two or more DAGs have the same ID, causing conflicts. |
Ensure that each DAG has a unique DAG ID. Review your
DAG files to identify any duplicate DAG IDs and resolve the conflict.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/d [...]
+| AERR089 | AttributeError | AttributeError
in PythonOperator | Occurs when the Python callable in a
PythonOperator is improperly defined or missing required attributes. |
Check the Python callable in your PythonOperator. Ensure that
all required arguments and attributes are defined and passed correctly.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/opera [...]
+| AERR090 | AirflowException | Error in callback
execution for TaskInstance | Occurs when a
failure callback defined for a task raises an error. |
Review the callback function for errors. Ensure that
the function is correctly defined and handles exceptions as expected.
|
https://airflow.apache.org/docs/apache-airflow/stable/advanced [...]
+| AERR091 | AirflowTaskTimeout | Operator
execution exceeded SLA | Happens when a task
operator runs longer than its Service Level Agreement (SLA). |
Review the task's execution time and optimize the code to ensure it
completes within the specified SLA. Adjust the SLA if necessary to account for
task duration. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/op [...]
+| AERR092 | AirflowDagNotFound | DAG not
found in scheduler | Occurs when a DAG file is not
loaded into the scheduler, often due to parsing errors or missing files. |
Check for errors in the DAG parsing logs. Ensure
that the DAG file exists in the DAGs folder and is properly formatted.
|
https://airflow.apache.org/docs/apache-airflow/stable/schedule [...]
+| AERR093 | AirflowException | Error in email
notification for task | Triggered when an
email notification for a task failure cannot be sent. |
Verify the email configuration in Airflow. Check if the SMTP
server is correctly configured and reachable from the Airflow instance.
|
https://airflow.apache.org/docs/apache-airflow/stable/advanced/alert [...]
+| AERR094 | AirflowWebServerException | Airflow
webserver won't start | Happens when the webserver
process fails, often due to misconfiguration or missing dependencies. |
Check the webserver logs for errors. Ensure that all
required dependencies are installed and the configuration is correct.
|
https://airflow.apache.org/docs/apache-airflow/stable/webse [...]
+| AERR095 | ModuleNotFoundError | No module named
'...' in DAG file | Occurs when the DAG
imports a Python module that is not installed or available. |
Install the missing Python module using pip or ensure
the module is available in the environment where Airflow is running.
|
https://airflow.apache.org/docs/apache-airflow/stable/howto/opera [...]
+| AERR096 | AirflowTemplateException | Missing template
field in operator | Happens when a required
templated field is not defined for a task operator. |
Review the task operator and ensure all required templated
fields are defined. Add any missing fields to the operator's parameters.
|
https://airflow.apache.org/docs/apache-airflow/stable/concepts/operat [...]
+| AERR097 | AirflowException | Error syncing
DAGs to remote storage | Happens when DAG files
fail to sync with a remote storage backend like S3 or GCS. |
Verify the connection to the remote storage backend. Ensure that the
correct credentials are configured and that the storage service is accessible.
|
https://airflow.apache.org/docs/apache-airflow/stable/storage.html#rem [...]
+| AERR098 | AirflowTaskTimeout | Subprocess in
task exceeded timeout | Happens when a
subprocess spawned by a task runs longer than its allowed time. |
Review the subprocess configuration and ensure it completes within
the task's timeout. If needed, increase the timeout or optimize the subprocess
execution. |
https://airflow.apache.org/docs/apache-airflow/stable/concepts/oper [...]
+| AERR099 | AirflowDatabaseException | Inconsistent
database schema | Occurs when the
metadata database schema is outdated or corrupted. |
Check the database schema and run any necessary
migrations. Ensure the database is in sync with the current Airflow version.
|
https://airflow.apache.org/docs/apache-airflow/stable/administration/ [...]
+| AERR100 | AirflowException | Invalid
DAG structure | Triggered when a
DAG's dependencies or attributes are incorrectly defined. |
Review the DAG structure for errors. Ensure that all
dependencies are correctly defined and that no circular dependencies exist.
|
https://airflow.apache.org/docs/apache-airflow/stable/concept [...]
