[
https://issues.apache.org/jira/browse/HADOOP-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993079#comment-13993079
]
Colin Patrick McCabe commented on HADOOP-10587:
-----------------------------------------------
Recently, I encountered an edit log with many {{DELEGATION_TOKEN_IDENTIFIER}}
log entries. Attempts to load this edit log failed with an out-of-memory
error, occurring in {{TokenIdentifier#getBytes}}. While the edit log seemed to
have an unusally large number of these log entries due to a misconfiguration, I
think it would also be useful to optimize {{TokenIdentifier#getBytes}} to
reduce this memory consumption somewhat.
> Use a thread-local cache in TokenIdentifier#getBytes to avoid creating many
> DataOutputBuffer objects
> ----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10587
> URL: https://issues.apache.org/jira/browse/HADOOP-10587
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.4.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-10587.001.patch
>
>
> We can use a thread-local cache in TokenIdentifier#getBytes to avoid creating
> many DataOutputBuffer objects. This will reduce our memory usage (for
> example, when loading edit logs), and help prevent OOMs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
