andrewfayres commented on a change in pull request #11204: Scala inference 
memory leak fix
URL: https://github.com/apache/incubator-mxnet/pull/11204#discussion_r194851374
 
 

 ##########
 File path: scala-package/core/src/main/scala/org/apache/mxnet/FeedForward.scala
 ##########
 @@ -224,13 +224,24 @@ class FeedForward private(
     var i = 0
     while (data.hasNext && i != numBatch) {
       val batch = data.next()
-      i += 1
-      ExecutorManager.loadData(batch, dataArrays)
-      predExec.forward(isTrain = false)
-      val padded = batch.pad
-      val realSize = batchSize - padded
-      for ((list, nd) <- outputs zip predExec.outputs) {
-        list += nd.slice(0, realSize).copy()
+      try {
+        i += 1
+        ExecutorManager.loadData(batch, dataArrays)
+        predExec.forward(isTrain = false)
+        val padded = batch.pad
+        val realSize = batchSize - padded
+        for ((list, nd) <- outputs zip predExec.outputs) {
+          // The slice is being written to a value so that dispose can be 
called after the copy.
+          // The one liner nd.slice().copy() leads to leaking the memory of 
the slice.
+          val ndSliced = nd.slice(0, realSize)
+          try {
+            list += ndSliced.copy()
+          } finally {
+            ndSliced.dispose()
+          }
+        }
+      } finally {
+        batch.dispose()
 
 Review comment:
   Summarizing the conversation we had offline: It appears that the copyTo in 
loadData is copying to memory in the Executor predExec. The memory in the 
executor seems to get reused across predict calls. I think that the proper 
place to dispose of this part would be to add a dispose method for FeedForward 
and have that dispose of it's predExec.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to