GitHub user dhruve opened a pull request:
https://github.com/apache/spark/pull/18392
[SPARK-21181] Release byteBuffers to suppress netty error messages
## What changes were proposed in this pull request?
We are explicitly calling release on the byteBuf's used to encode the
string to Base64 to suppress the memory leak error message reported by netty.
This is to make it less confusing for the user.
## changes proposed in this fix
By explicitly invoking release on the byteBuf's we are decrement the
internal reference counts for the wrappedByteBuf's. Now, when the GC kicks in,
these would be reclaimed as before, just that netty wouldn't report any memory
leak error messages as the internal ref. counts are now 0.
## How was this patch tested?
Ran a few spark-applications and examined the logs. The error message no
longer appears.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dhruve/spark bug/SPARK-21181
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18392.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18392
----
commit 21b4c0b26899417b4d4a562997b1d9adeac51081
Author: Dhruve Ashar <[email protected]>
Date: 2017-06-22T16:37:55Z
[SPARK-21181] Release byteBuffers to suppress netty error messages
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
