[
https://issues.apache.org/jira/browse/TEZ-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052370#comment-16052370
]
Jason Lowe commented on TEZ-3741:
---------------------------------
bq. may be we need a similar change for PipelinedSorter (SortSpan.close()) for
completeness. Thoughts?
I'm not an expert on the PipelinedSorter, but I think we're OK there. It looks
like it only tracks one SortSpan at a time, so we're effectively freeing it
when we unreference the old span when moving to the next span.
bq. Do we want it marked as @VisibleForTesting or have it available for other
src/main files in the package?
I don't think there's an issue if something else in the package wants to use
this method for something besides testing, but I highly doubt it will happen in
practice. I personally don't think we need the annotation but if you feel it's
useful/necessary I can add it.
> Tez outputs should free memory when closed
> ------------------------------------------
>
> Key: TEZ-3741
> URL: https://issues.apache.org/jira/browse/TEZ-3741
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.1, 0.9.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-3741.001.patch, TEZ-3741.002.patch
>
>
> Memory buffers aren't being released as quickly as they could be, e.g.:
> DefaultSorter is holding onto the very large kvbuffer byte array even after
> close() is called, and Ordered and Unordered outputs should remove references
> to sorter and kvWriter in their close.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
