wes-devore-mpulse opened a new issue, #31784:
URL: https://github.com/apache/airflow/issues/31784
### Apache Airflow version
2.6.1
### What happened
We want to use
"import ipdb; ipdb.set_trace()"
in the task execution thread (the python callable)
This just broke when upgrading from 2.4.0 to 2.6.1.
Now we get the following stack trace:
Traceback (most recent call last):
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/application/application.py",
line 955, in run
loop = asyncio.get_event_loop()
File "/usr/local/lib/python3.10/asyncio/events.py", line 656, in
get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread
'ThreadPoolExecutor-0_0'.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/airflow/__main__.py", line
48, in main
args.func(args)
File "/usr/local/lib/python3.10/site-packages/airflow/cli/cli_config.py",
line 51, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/cli.py", line
112, in wrapper
return f(*args, **kwargs)
File
"/usr/local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py",
line 614, in task_test
ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
File "/usr/local/lib/python3.10/site-packages/airflow/utils/session.py",
line 76, in wrapper
return func(*args, session=session, **kwargs)
File
"/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line
1721, in run
self._run_raw_task(
File "/usr/local/lib/python3.10/site-packages/airflow/utils/session.py",
line 73, in wrapper
return func(*args, **kwargs)
File
"/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line
1407, in _run_raw_task
self._execute_task_with_callbacks(context, test_mode)
File
"/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line
1558, in _execute_task_with_callbacks
result = self._execute_task(context, task_orig)
File
"/usr/local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line
1623, in _execute_task
result = execute_callable(context=context)
File
"/usr/local/lib/python3.10/site-packages/airflow/operators/python.py", line
181, in execute
return_value = self.execute_callable()
File
"/usr/local/lib/python3.10/site-packages/airflow/operators/python.py", line
198, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/usr/local/airflow/dags/factory/legacy_tasks.py", line 32, in
get_files
local_files_path, files = report_object.get_files()
File "/usr/local/airflow/dags/factory/legacy_tasks.py", line 32, in
get_files
local_files_path, files = report_object.get_files()
File "/usr/local/lib/python3.10/bdb.py", line 90, in trace_dispatch
return self.dispatch_line(frame)
File "/usr/local/lib/python3.10/bdb.py", line 114, in dispatch_line
self.user_line(frame)
File "/usr/local/lib/python3.10/pdb.py", line 253, in user_line
self.interaction(frame, None)
File
"/home/astro/.local/lib/python3.10/site-packages/IPython/core/debugger.py",
line 335, in interaction
OldPdb.interaction(self, frame, traceback)
File "/usr/local/lib/python3.10/pdb.py", line 348, in interaction
self._cmdloop()
File "/usr/local/lib/python3.10/pdb.py", line 313, in _cmdloop
self.cmdloop()
File
"/home/astro/.local/lib/python3.10/site-packages/IPython/terminal/debugger.py",
line 133, in cmdloop
).result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in
result
return self.__get_result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in
__get_result
raise self._exception
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in
run
result = self.fn(*self.args, **self.kwargs)
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/shortcuts/prompt.py",
line 1035, in prompt
return self.app.run(
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/application/application.py",
line 958, in run
return asyncio.run(coro)
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in
run_until_complete
return future.result()
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/application/application.py",
line 875, in run_async
return await _run_async(f)
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/application/application.py",
line 732, in _run_async
self._request_absolute_cursor_position()
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/application/application.py",
line 1215, in _request_absolute_cursor_position
self.renderer.request_absolute_cursor_position()
File "/usr/local/lib/python3.10/site-packages/prompt_toolkit/renderer.py",
line 506, in request_absolute_cursor_position
do_cpr()
File "/usr/local/lib/python3.10/site-packages/prompt_toolkit/renderer.py",
line 495, in do_cpr
self.output.ask_for_cpr()
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/output/vt100.py", line
711, in ask_for_cpr
self.flush()
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/output/vt100.py", line
704, in flush
flush_stdout(self.stdout, data)
File
"/usr/local/lib/python3.10/site-packages/prompt_toolkit/output/flush_stdout.py",
line 33, in flush_stdout
stdout.buffer.write(data.encode(stdout.encoding or "utf-8", "replace"))
AttributeError: 'NoneType' object has no attribute 'write'
If you suspect this is an IPython 8.14.0 bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at [email protected]
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
### What you think should happen instead
We should get into the ipdb debugging shell.
### How to reproduce
1. place the following code in a python callable in an airflow task `import
ipdb; ipdb.set_trace()`
2. Run the task from the airflow cli `airflow tasks test <dag_id> <task_id>
<run_id>
3. See the stack trace
4. We should not get an error, but be in the ipdb debug shell.
### Operating System
Debian 11.3
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else
I reported this exact same bug back in 2.3.4, and it was fixed, but it has
come back. Can we please add tests to make sure debuggers like ipdb won't break
with future releases?
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
