GitHub user komalrajputtrootech edited a comment on the discussion: Task exited
with return code 1 with no errors
Any idea what is the use of DJANGO_ENVIRONMENT_HAS_BEEN_SETUP variable. Is it
set anywhere else other than operators code ?
I have done somewhat similar to that in github repo that you shared:
```
class DjangoOperator(PythonOperator):
"""
An Airflow Operator for running any Python function that requires the
Django environment.
This operator ensures:
- Django project path is added to `sys.path`.
- Django settings module is loaded.
- `django.setup()` is called before running the task.
- Allows specifying the callable either directly as a Python function,
or via a dotted string path (e.g. `"myapp.tasks.my_function"`).
"""
def __init__(
self,
*,
python_callable: Callable | str,
op_args: Collection[Any] | None = None,
op_kwargs: Mapping[str, Any] | None = None,
templates_dict: dict[str, Any] | None = None,
templates_exts: Sequence[str] | None = None,
show_return_value_in_logs: bool = True,
**kwargs,
) -> None:
# Bypass parent's python_callable check
self._python_callable = python_callable
super().__init__(
python_callable=lambda: None, # Bypass the AirflowException check
op_args=op_args,
op_kwargs=op_kwargs,
templates_dict=templates_dict,
templates_exts=templates_exts,
show_return_value_in_logs=show_return_value_in_logs,
**kwargs,
)
@staticmethod
def setup_django():
"""
Sets up Django so that the ORM and other Django-specific components can
be used
within Airflow tasks.
- Adds the Django project root to `sys.path`.
- Configures the `DJANGO_SETTINGS_MODULE` environment variable.
- Calls `django.setup()` to initialize Django.
"""
# Absolute path to the Django project root where `manage.py` is located
django_root = "path_to_project"
if django_root not in sys.path:
sys.path.insert(0, django_root)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
# Initialize the Django application
django.setup()
def resolve_callable(self):
"""
Resolves the python_callable into a function object.
Returns
-------
callable
The resolved Python function.
Raises
------
TypeError
If python_callable is neither a function nor a dotted string path.
"""
if callable(self._python_callable):
return self._python_callable
elif isinstance(self._python_callable, str):
module_path, func_name = self._python_callable.rsplit(".", 1)
module = importlib.import_module(module_path)
return getattr(module, func_name)
else:
raise TypeError(
"`python_callable` must be a callable or dotted string path"
)
def pre_execute(self, context):
"""
Prepares Django and resolves the Python callable before the task runs.
"""
self.setup_django()
self.python_callable = self.resolve_callable()
```
GitHub link:
https://github.com/apache/airflow/discussions/54947#discussioncomment-14256505
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
