New submission from Arkady M <lary...@gmail.com>:
I am running an HTTP server (socketserver.ThreadingMixIn,
http.server.HTTPServer) in a Docker container (FROM ubuntu:19.10)
Occasionally I get an exception:
Exception happened during processing of request from ('172.17.0.1', 35756)
Traceback (most recent call last):
File "/usr/lib/python3.7/socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "service.py", line 221, in __init__
super(UrlExtractorServer, self).__init__(*args, **kwargs)
File "/usr/lib/python3.7/socketserver.py", line 720, in __init__
self.handle()
File "/usr/lib/python3.7/http/server.py", line 426, in handle
self.handle_one_request()
File "/usr/lib/python3.7/http/server.py", line 414, in handle_one_request
method()
File "service.py", line 488, in do_POST
self._post_extract(url)
File "service.py", line 459, in _post_extract
extracted_links, err_msg = self._extract_links(transaction_id,
attachment_id, zip_password, data)
File "service.py", line 403, in _extract_links
error, results = call_timeout(process_deadline,
extractor.extract_links_binary_multiprocess, args=data)
File "service.py", line 175, in call_timeout
manager = multiprocessing.Manager()
File "/usr/lib/python3.7/multiprocessing/context.py", line 56, in Manager
m.start()
File "/usr/lib/python3.7/multiprocessing/managers.py", line 563, in start
self._process.start()
File "/usr/lib/python3.7/multiprocessing/process.py", line 111, in start
_cleanup()
File "/usr/lib/python3.7/multiprocessing/process.py", line 56, in _cleanup
if p._popen.poll() is not None:
AttributeError: 'NoneType' object has no attribute 'poll'
I am in the process of preparingof of a reasonably simple piece of code
demonstrating the problem.
Meanwhile the following can be important. In the code below I ma getting the
elapse < timeout (20 times from 70K). In all case psutil.Process() returned
psutil.NoSuchProcess
time_start = time.time()
job = multiprocessing.Process(target=func, args=(args, results),
kwargs=kwargs)
job.start()
job.join(timeout)
elapsed = time.time()-time_start
if job.is_alive():
try:
process = psutil.Process(job.pid)
process_error = f"pid {job.pid} status {process.status} {process}"
except Exception as e:
process_error = f"psutil.Process() failed {e}"
if elapsed < timeout:
print("elapsed < timeout")
----------
messages: 370695
nosy: Arkady M
priority: normal
severity: normal
status: open
title: Exception in multiprocessing/context.py under load
type: crash
versions: Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue40860>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
