curcur edited a comment on pull request #13648:
URL: https://github.com/apache/flink/pull/13648#issuecomment-713341581
Thanks @rkhachatryan so much for the great question on how different threads
access the same view (in other words, threading model in netty + task thread
interaction on view). Thanks @zhijiangW a lot for the explanation of the
underlying threading model based on offline-discussion and great insights.
I've spent some time on different paths of how the view is created/released
in different scenarios, here is what I think is happening. @zhijiangW , please
correct me if I miss anything, thanks!
For a streaming job, each `view` is created through
PartitionRequestServerHandler (one of the netty worker threads)'s
PartitionRequest. The created view is owned by subPartition, and a reference of
the view is returned and bounded with a reader. Reader belongs to
PartitionRequestServerHandler.
`PipelinedApproximateSubpartition#releaseView` is used in two places:
1. PipelinedApproximateSubpartition#createView (From Netty thread, it can
see the current version of view belonging to the subpartition)
2. PipelinedApproximateSubpartition#releaseAllResources,
This is called from two places,
- one from `reader`, the reader uses its own reference of the view to
release resources. This is from downstream task cancelation.
- one from `subPartition release`. This is fine, the subpartition
releases its own view.
Two questions:
1. Is the old view continue to read data if not disposed of successfully
before the new view is created?
- No, because the reader and the view would be removed up downstream task's
cancelation request;
2. What if the netty thread1 release view after netty thread2 recreates the
view?
thread2 release the view thread1 holds the reference before creating a new
view.
When thread1 ties to release the view from the old reference, as long as we
disconnects the connection between the old view and the subpartition, we should
be fine.
In that sense, we can simply set the view.parent = null when releasing the
view.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
