[
https://issues.apache.org/jira/browse/ARROW-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990886#comment-16990886
]
David Li edited comment on ARROW-7343 at 12/8/19 3:28 PM:
----------------------------------------------------------
Found via debugging an internal application:
# Client cancels a doGet.
# Server doesn't observe this and calls putNext. (or, the cancellation happens
in between when the server calls putNext and the client cancels.)
# An ArrowMessage is created, retaining references to buffers. The references
won't be cleared until {{ArrowMessage#asInputStream}} is called.
# {{ServerCallStreamObserverImpl#onNext}} finds that the call has been
cancelled, _and Flight has set an onCancel handler_ in
{{FlightService.GetListener.<constructor>}}. Thus, it doesn't throw, and
returns early (_before_ calling {{ServerCallImpl#sendMessage}}).
# Thus, the message is never freed.
was (Author: lidavidm):
Found via debugging an internal application:
#. Client cancels a doGet.
#. Server doesn't observe this and calls putNext. (or, the cancellation happens
in between when the server calls putNext and the client cancels.)
#. An ArrowMessage is created, retaining references to buffers. The references
won't be cleared until {{ArrowMessage#asInputStream}} is called.
#. {{ServerCallStreamObserverImpl#onNext}} finds that the call has been
cancelled, _and Flight has set an onCancel handler_ in
{{FlightService.GetListener.<constructor>}}. Thus, it doesn't throw, and
returns early (_before_ calling {{ServerCallImpl#sendMessage}}).
#. Thus, the message is never freed.
> Memory leak in Flight DoGet when client cancels
> -----------------------------------------------
>
> Key: ARROW-7343
> URL: https://issues.apache.org/jira/browse/ARROW-7343
> Project: Apache Arrow
> Issue Type: Bug
> Components: FlightRPC, Java
> Affects Versions: 0.14.0
> Reporter: David Li
> Assignee: David Li
> Priority: Major
>
> I believe this causes things like ARROW-4765.
> If a stream is interrupted or otherwise not drained on the server-side, the
> serialized form of the ArrowMessage (DrainableByteBufInputStream) will sit
> around forever, leaking memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
