PApostol commented on issue #8212:
URL: https://github.com/apache/airflow/issues/8212#issuecomment-1361128280
Hi @potiuk, we've also had the same issue with S3 remote logs and I've tried
out your fix. While the fix succeeds in fetching the logs from S3 on the UI, it
seems to break the scheduler upon restart. The error is reproducible every time
I try to start the scheduler after the fix is applied (note that webserver
works fine). Removing the fix from `airflow/__init__.py` resolves the error.
I'm attaching the traceback below. Using Airflow: 2.3.4, Python 3.7.3,
LocalExecutor.
```
Traceback (most recent call last):
File "/home/ec2-user/venv/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/__main__.py", line 38,
in main
args.func(args)
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py",
line 51, in command
return func(*args, **kwargs)
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/utils/cli.py", line
99, in wrapper
return f(*args, **kwargs)
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py",
line 76, in scheduler
_run_scheduler_job(args=args)
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py",
line 46, in _run_scheduler_job
job.run()
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/jobs/base_job.py",
line 244, in run
self._execute()
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 743, in _execute
self.executor.start()
File
"/home/ec2-user/venv/lib/python3.7/site-packages/airflow/executors/local_executor.py",
line 348, in start
self.result_queue = self.manager.Queue()
File "/usr/lib64/python3.7/multiprocessing/managers.py", line 724, in temp
token, exp = self._create(typeid, *args, **kwds)
File "/usr/lib64/python3.7/multiprocessing/managers.py", line 607, in
_create
conn = self._Client(self._address, authkey=self._authkey)
File "/usr/lib64/python3.7/multiprocessing/connection.py", line 498, in
Client
answer_challenge(c, authkey)
File "/usr/lib64/python3.7/multiprocessing/connection.py", line 742, in
answer_challenge
message = connection.recv_bytes(256) # reject large message
File "/usr/lib64/python3.7/multiprocessing/connection.py", line 216, in
recv_bytes
buf = self._recv_bytes(maxlength)
File "/usr/lib64/python3.7/multiprocessing/connection.py", line 407, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib64/python3.7/multiprocessing/connection.py", line 379, in
_recv
chunk = read(handle, remaining)
BlockingIOError: [Errno 11] Resource temporarily unavailable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
