dstandish opened a new issue, #47567:
URL: https://github.com/apache/airflow/issues/47567
### Body
Hey team :slightly_smiling_face:
Is the GitDagBundle working yet? And if so, what am I missing?
I set up a git_default connection with content read permissions for all my
repos and this config:
[dag_processor]
dag_bundle_config_list=[{"name": "dags-folder", "classpath":
"airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {}}, {"name":
"tjanif/dynamic-task-mapping-tutorial", "classpath":
"airflow.dag_processing.bundles.git.GitDagBundle", "kwargs": {"subdir": "dags",
"tracking_ref": "main", "refresh_interval": 30}}]
and my dag-processor crashes with:
dag-processor | FileNotFoundError: [Errno 2] No such file or directory:
'/var/folders/hm/tgdl9dqs2j18vhrsk7s6c1q80000gn/T/airflow/dag_bundles/_locks/tjanif/dynamic-task-mapping-tutorial.lock'
directly after starting it. (Full error in :thread: )
```
dag-processor | [2025-03-10T12:04:34.805+0100] {manager.py:246} INFO -
Checking for new files in bundle dags-folder every 300 seconds
dag-processor | [2025-03-10T12:04:34.805+0100] {manager.py:246} INFO -
Checking for new files in bundle tjanif/dynamic-task-mapping-tutorial every 30
seconds
dag-processor | [2025-03-10T12:04:34.806+0100] {manager.py:504} INFO -
Refreshing bundle dags-folder
dag-processor | [2025-03-10T12:04:34.807+0100] {manager.py:554} INFO -
Searching for files in dags-folder at
/Users/tamara.fingerlin/airflow/dags_manning
dag-processor | [2025-03-10T12:04:34.815+0100] {manager.py:556} INFO - Found
6 files for bundle dags-folder
dag-processor | [2025-03-10T12:04:34.823+0100]
{dag_processor_job_runner.py:63} ERROR - Exception when executing
DagProcessorJob
dag-processor | Traceback (most recent call last):
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/jobs/dag_processor_job_runner.py",
line 61, in _execute
dag-processor | self.processor.run()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 252, in run
dag-processor | return self._run_parsing_loop()
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 323, in _run_parsing_loop
dag-processor | self._refresh_dag_bundles(known_files=known_files)
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 474, in _refresh_dag_bundles
dag-processor | bundle.initialize()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/git.py",
line 205, in initialize
dag-processor | self._initialize()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/git.py",
line 186, in _initialize
dag-processor | with self.lock():
dag-processor | ^^^^^^^^^^^
dag-processor | File
"/Users/tamara.fingerlin/.pyenv/versions/3.12.8/lib/python3.12/contextlib.py",
line 137, in __enter__
dag-processor | return next(self.gen)
dag-processor | ^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/base.py",
line 338, in lock
dag-processor | with open(lock_file_path, "w") as lock_file:
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | FileNotFoundError: [Errno 2] No such file or directory:
'/var/folders/hm/tgdl9dqs2j18vhrsk7s6c1q80000gn/T/airflow/dag_bundles/_locks/tjanif/dynamic-task-mapping-tutorial.lock'
dag-processor | Traceback (most recent call last):
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/bin/airflow", line 12, in <module>
dag-processor | sys.exit(main())
dag-processor | ^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/__main__.py", line 58, in main
dag-processor | args.func(args)
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 49,
in command
dag-processor | return func(*args, **kwargs)
dag-processor | ^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/utils/cli.py", line 112, in
wrapper
dag-processor | return f(*args, **kwargs)
dag-processor | ^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function
dag-processor | return func(*args, **kwargs)
dag-processor | ^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/cli/commands/local_commands/dag_processor_command.py",
line 54, in dag_processor
dag-processor | run_command_with_daemon_option(
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/cli/commands/local_commands/daemon_utils.py",
line 86, in run_command_with_daemon_option
dag-processor | callback()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/cli/commands/local_commands/dag_processor_command.py",
line 57, in <lambda>
dag-processor | callback=lambda: run_job(job=job_runner.job,
execute_callable=job_runner._execute),
dag-processor |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/utils/session.py", line 101,
in wrapper
dag-processor | return func(*args, session=session, **kwargs)
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/jobs/job.py", line 342, in
run_job
dag-processor | return execute_job(job, execute_callable=execute_callable)
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/jobs/job.py", line 371, in
execute_job
dag-processor | ret = execute_callable()
dag-processor | ^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/jobs/dag_processor_job_runner.py",
line 61, in _execute
dag-processor | self.processor.run()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 252, in run
dag-processor | return self._run_parsing_loop()
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 323, in _run_parsing_loop
dag-processor | self._refresh_dag_bundles(known_files=known_files)
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 474, in _refresh_dag_bundles
dag-processor | bundle.initialize()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/git.py",
line 205, in initialize
dag-processor | self._initialize()
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/git.py",
line 186, in _initialize
dag-processor | with self.lock():
dag-processor | ^^^^^^^^^^^
dag-processor | File
"/Users/tamara.fingerlin/.pyenv/versions/3.12.8/lib/python3.12/contextlib.py",
line 137, in __enter__
dag-processor | return next(self.gen)
dag-processor | ^^^^^^^^^^^^^^
dag-processor | File "/Users/tamara.fingerlin/0_PARA/Projects/Airflow3.0/a
wheel/beta2/lib/python3.12/site-packages/airflow/dag_processing/bundles/base.py",
line 338, in lock
dag-processor | with open(lock_file_path, "w") as lock_file:
dag-processor | ^^^^^^^^^^^^^^^^^^^^^^^^^
dag-processor | FileNotFoundError: [Errno 2] No such file or directory:
'/var/folders/hm/tgdl9dqs2j18vhrsk7s6c1q80000gn/T/airflow/dag_bundles/_locks/tjanif/dynamic-task-mapping-tutorial.lock'
```
@ephraimbuddy @jedcunningham
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow
project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
