raulcd commented on code in PR #50687:
URL: https://github.com/apache/arrow/pull/50687#discussion_r3673186375
##########
python/pyarrow/tests/test_flight.py:
##########
@@ -1128,6 +1130,43 @@ def test_flight_server_location_argument():
assert isinstance(server, FlightServerBase)
+# The following tests are for GH-50684, which was a memory leak
+# in FlightServerBase.
+def test_flight_server_is_freed():
+ # Calling server.shutdown manually should free the server object.
+ server = FlightServerBase('grpc://localhost:0')
+ server.shutdown()
+ server.wait()
+ ref = weakref.ref(server)
+ del server
+ gc.collect()
+ assert ref() is None
+
+
+def test_flight_server_is_freed_on_exit():
+ # Using FlightServerBase as a context manager should free
+ # the server object on exit.
+ with FlightServerBase('grpc://localhost:0') as server:
+ ref = weakref.ref(server)
+ del server
+ gc.collect()
+ assert ref() is None
+
+
[email protected](
+ reason="GH-50684: FlightServerBase is not freed on delete without shutdown"
+)
+def test_flight_server_is_freed_without_shutdown():
+ # GH-50684: Not calling server.shutdown() currently leaks the server
object.
+ # This test is expected to fail until the issue is fixed but is included
+ # for completeness and further discussion.
Review Comment:
I still have to finish grasping my head around this but while a weakref
solves those lifetime issues we would introduce a deadlock.
From my understanding we still require a strong reference to the server for
the dispatch methods in order to guarantee the server is present while serving.
If the Python server user reference disappears during the dispatch method
execution, once the dispatch method finishes execution refcount reaches 0 and
the finalizer runs on the serving thread, calling shutdown.
We seem to point that we should not call shutdown from a serving thread as
this could block forever:
https://github.com/apache/arrow/blob/e1dfbefe2ebfe610609f4d718f652b910c966521/python/pyarrow/_flight.pyx#L3219-L3228
I can't see this prohibition on the C++ docs stated specifically but we hint
that this blocks until requests are finished, from my understanding the current
request won't finish because it is the one calling shutdown.
https://github.com/apache/arrow/blob/e1dfbefe2ebfe610609f4d718f652b910c966521/cpp/src/arrow/flight/server.h#L225-L232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
