[issue43329] Multiprocessing Manager Client Not Reconnecting

[Michael L. Boom]

Michael L. Boom  added the comment:

Is there anything that I can do, or the company I work for can do, to get a 
developer assigned to fix this bug?  It is hard to use multiprocessing remote 
method calls without this bug being fixed.  The software wouldn't be robust 
enough for production.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43329] Multiprocessing Manager Client Not Reconnecting

[Ethan Furman]

Ethan Furman  added the comment:

Here is the test, reduced and in a single script:

--- 8< 
import multiprocessing.managers, os, sys, time

if __name__ == '__main__':
address = ("127.0.0.1", 54321)

class TestManager(multiprocessing.managers.BaseManager):
pass

if sys.argv[1] == 'server':

class TestClass(object):
def test_method(self):
print("In test_method")
return "TEST"

TestManager.register("Test", TestClass)
manager = TestManager(address = address, authkey = 
"1234".encode("utf-8"))
print('waiting...')
manager.get_server().serve_forever()

else:

TestManager.register("Test")

manager = TestManager(address = address, authkey = 
"1234".encode("utf-8"))
manager.connect()
test_class = manager.Test()

print(test_class.test_method())

print("Kill and restart the server and press return")
sys.stdin.readline()

try:
print(test_class.test_method())
except EOFError:
print('EOF received\n')

# reestablish connection
manager.connect()
test_class = manager.Test()

# trigger error
print(test_class.test_method())
--- 8< 

Running it in two terminals gives (only showing client side):

---
$ ./python ~/test_mp client
TEST
Kill and restart the server and press return
# hit 
EOF received

Traceback (most recent call last):
  File "/home/ethan/test_mp", line 45, in 
print(test_class.test_method())
  File "", line 2, in test_method
  File "/source/virtualenv/lib/python3.9/multiprocessing/managers.py", line 
808, in _callmethod
conn.send((self._id, methodname, args, kwds))
  File "/source/virtualenv/lib/python3.9/multiprocessing/connection.py", line 
211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
  File "/source/virtualenv/lib/python3.9/multiprocessing/connection.py", line 
416, in _send_bytes
self._send(header + buf)
  File "/source/virtualenv/lib/python3.9/multiprocessing/connection.py", line 
373, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
---

The problem appears to be that the call to `manager.connect()` after the EOF is 
not getting a new connection, but is reusing the old one (?).  This is as far 
as I can take this; hopefully somebody with more multiprocessing/socket skills 
can take it from here.

--
nosy: +davin, pitrou, serhiy.storchaka
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43329] Multiprocessing Manager Client Not Reconnecting

[Michael L. Boom]

Michael L. Boom  added the comment:

I don't think I even got this working on the small scale.  The unit test in the 
bug report is about as small as it gets and it doesn't work.  The issue still 
exists in Python 3.9.5.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43329] Multiprocessing Manager Client Not Reconnecting

[Ethan Furman]

Ethan Furman  added the comment:

Michael Boon wrote:
---
> The [above] issue is pretty serious and it is preventing me from using a
> system I wrote on a larger scale.

That statement suggests you were able to get this working on a small scale -- 
is that correct?  What does the working small-scale code look like?

--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43329] Multiprocessing Manager Client Not Reconnecting

[Michael L. Boom]

New submission from Michael L. Boom :

The client doesn't reconnect automatically, or explicitly.  I just get 
BrokenPipeError over and over.


Manager:
import multiprocessing.managers, os, sys, time
class TestClass(object):
def test_method(self):
print("In test_method")
return "TEST"

class TestManager(multiprocessing.managers.BaseManager):
pass

address = ("127.0.0.1", 54321)
TestManager.register("Test", TestClass)
manager = TestManager(address = address, authkey = "1234".encode("utf-8"))
manager.get_server().serve_forever()


Client:
import multiprocessing.managers, os, sys, time

class TestManager(multiprocessing.managers.BaseManager):
pass

address = ("127.0.0.1", 54321)
TestManager.register("Test")
manager = TestManager(address = address, authkey = "1234".encode("utf-8"))
manager.connect()
test_class = manager.Test()

def call_it():
time.sleep(1)
result = test_class.test_method()
print("result: '" + str(type(result)) + ", " + str(result) + "'")

call_it()

print("Kill and restart the server and press return")
sys.stdin.readline()

error = False
while (True):
try:
if (error):
print("Reconnecting")
manager.connect()
test_class = manager.Test()
call_it()
error = False
except Exception as e:
print("Got exception " + str(type(e)) + ", " + repr(e))
error = True
time.sleep(1)

--
components: Library (Lib)
messages: 387726
nosy: boom0192
priority: normal
severity: normal
status: open
title: Multiprocessing Manager Client Not Reconnecting
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com