Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/5743#discussion_r29510060
--- Diff:
network/common/src/main/java/org/apache/spark/network/server/OneForOneStreamManager.java
---
@@ -80,12 +95,17 @@ public ManagedBuffer getChunk(long streamId, int
chunkIndex) {
}
@Override
- public void connectionTerminated(long streamId) {
- // Release all remaining buffers.
- StreamState state = streams.remove(streamId);
- if (state != null && state.buffers != null) {
- while (state.buffers.hasNext()) {
- state.buffers.next().release();
+ public void connectionTerminated(Channel channel) {
+ // Close all streams which have been associated with the channel.
+ for (Map.Entry<Long, StreamState> entry: streams.entrySet()) {
+ StreamState state = entry.getValue();
+ if (state.associatedChannel == channel) {
+ streams.remove(entry.getKey());
--- End diff --
Ah, good point, this is safe because our Map is a ConcurrentHashMap (or
else you would need to use an iterator to remove it safely). Would you mind
making the left-hand type of the declaration of `streams` a
`ConcurrentHashMap`? This is not the first place where we rely on the semantics
of a ConcurrentHashMap over a general Map, and we should use proper style
therefore.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
