Github user eyalfa commented on the issue:
https://github.com/apache/spark/pull/21369
@advancedxy , using jvisualvm+heap dump I could see that the second
introduced test case ("drop all references to the underlying map once the
iterator is exhausted") eliminated all references to the underlying map:
heap contained one instance of `SizeTrackingAppendOnlyMap`, but all
references to it where unreachable, hence it was due to be evicted.
found one instance of CompletionIterator (actually anonymous class deriving
it) which had references to the `SpillableIterator`, direct ref as member 'sub'
and another one via member `completionFunction$1`.
the `SpillableIterator` had a single ref to the `ExternalAppendOnlyMap`
which already had its `currentMap` field already nullified.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
