Repository: spark Updated Branches: refs/heads/branch-1.4 82be68f10 -> 62308097b
[SPARK-7490] [CORE] [Minor] MapOutputTracker.deserializeMapStatuses: close input streams GZIPInputStream allocates native memory that is not freed until close() or when the finalizer runs. It is best to close() these streams explicitly. stephenh made the same change for serializeMapStatuses in commit b0d884f0. This is the same change for deserialize. (I ran the unit test suite! it seems to have passed. I did not make a JIRA since this seems "trivial", and the guidelines suggest it is not required for trivial changes) Author: Evan Jones <[email protected]> Closes #5982 from evanj/master and squashes the following commits: 0d76e85 [Evan Jones] [CORE] MapOutputTracker.deserializeMapStatuses: close input streams (cherry picked from commit 25889d8d97094325f10fbf52f3b36412f212eeb2) Signed-off-by: Sean Owen <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/62308097 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/62308097 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/62308097 Branch: refs/heads/branch-1.4 Commit: 62308097b2a7376f7d2e3c3c28fa694125c9bddf Parents: 82be68f Author: Evan Jones <[email protected]> Authored: Fri May 8 22:00:39 2015 +0100 Committer: Sean Owen <[email protected]> Committed: Fri May 8 22:01:01 2015 +0100 ---------------------------------------------------------------------- core/src/main/scala/org/apache/spark/MapOutputTracker.scala | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/62308097/core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/MapOutputTracker.scala b/core/src/main/scala/org/apache/spark/MapOutputTracker.scala index 1607228..0184228 100644 --- a/core/src/main/scala/org/apache/spark/MapOutputTracker.scala +++ b/core/src/main/scala/org/apache/spark/MapOutputTracker.scala @@ -367,7 +367,11 @@ private[spark] object MapOutputTracker extends Logging { // Opposite of serializeMapStatuses. def deserializeMapStatuses(bytes: Array[Byte]): Array[MapStatus] = { val objIn = new ObjectInputStream(new GZIPInputStream(new ByteArrayInputStream(bytes))) - objIn.readObject().asInstanceOf[Array[MapStatus]] + Utils.tryWithSafeFinally { + objIn.readObject().asInstanceOf[Array[MapStatus]] + } { + objIn.close() + } } // Convert an array of MapStatuses to locations and sizes for a given reduce ID. If --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
