[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Antoine Pitrou]

Antoine Pitrou added the comment:

I think this issue is mistaken.  The reader and writer objects are closed 
automatically when they are destroyed (see Connection.__del__).  The only thing 
that may lack is a way to close them more eagerly.

In any case, I'm closing as a duplicate of issue 30966.

--
dependencies:  -multiprocessing.queues.SimpleQueue leaks 2 fds
nosy: +pitrou
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> multiprocessing.queues.SimpleQueue leaks 2 fds

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Xiang Zhang]

Changes by Xiang Zhang :


--
dependencies: +multiprocessing.queues.SimpleQueue leaks 2 fds

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Josh Rosenberg]

Changes by Josh Rosenberg :


--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Camilla Montonen]

Camilla Montonen added the comment:

I did some investigating using a test script and Python 3.7.0a0

from multiprocessing import Pool
import os
import time

def f(x):
time.sleep(30)
return x*x

if __name__=='__main__':
print('Main pid {0}'.format(os.getpid()))
p = Pool(5)
p.map(f, [1,2,3])
print('Returned')
time.sleep(30)
and grepping for pipe and the parentpid in the output from lsof ( lsof | grep 
python.*.*pipe ). The pipes opened at the start of the script are 
still open even after the line print('Returned') is executed. I suppose this is 
expected because I did not call *p.close()*. All pipes are cleaned up after the 
parent process finishes.

When I repeat the experiment calling p.close() after p.map returns, all that is 
left is the 9 pipes opened by the parent. All pipes are cleaned up after parent 
script exits. 

@shani - could you please clarify how you were able to detect the leaking pipes?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Camilla Montonen]

Changes by Camilla Montonen :


--
nosy: +Winterflower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Charles-François Natali]

Charles-François Natali added the comment:

Interestingly, there is no close() method on SimpleQueue...

--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23267
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +davin, sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23267
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23267] multiprocessing pool.py doesn't close inqueue and outqueue pipes on termination

[shani]

New submission from shani:

Multiprocessing pool.py gets SimpleQueue objects as inqueue and outqueue.
when it terminates, it doesn't call the close() method of the queues' readers 
and writers.
As a results, 4 file pipes leak in one pool termination.

Expected: The pool closes reader and writer pipes of the inqueue and outqueue 
when it terminates.
What did happen: the pool doesn't close the pipes. 4 pipes leak.

--
components: Library (Lib)
messages: 234244
nosy: shanip
priority: normal
severity: normal
status: open
title: multiprocessing pool.py doesn't close inqueue and outqueue pipes on 
termination
type: resource usage

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23267
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com