[
https://issues.apache.org/jira/browse/STORM-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051255#comment-15051255
]
ASF GitHub Bot commented on STORM-1206:
---------------------------------------
Github user zhuoliu commented on the pull request:
https://github.com/apache/storm/pull/886#issuecomment-163689068
Hi Fang, @hustfxj. Great question! We also met the same problem. Actually
our setting for gc log name is "-Xloggc:artifacts/gc.%p.log". Then gc log won't
be overwritten due to worker restating.
Then we will also need to https://issues.apache.org/jira/browse/STORM-1206
to avoid too many gc files' names crashing the logviewer JVM (this might happen
in a very very extreme and rare case, only once in our cluster).
> Reduce logviewer memory usage
> -----------------------------
>
> Key: STORM-1206
> URL: https://issues.apache.org/jira/browse/STORM-1206
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Zhuo Liu
> Assignee: Zhuo Liu
> Priority: Minor
>
> In production, we ran into an issue with logviewers bouncing with out of
> memory errors. Note that this happens very rarely, we met this in some
> extreme case when super frequently restarting of workers generates a huge
> number of gc files (~1M files).
> What was happening is that if there are lots of log files (~1 M files) for a
> particular headless user, we would have so many strings resident in memory
> that logviewer would run out of heap space.
> We were able to work around this by increasing the heap space, but we should
> consider putting some sort of an upper bound on the number of files so that
> we don't run in to this issue, even with the bigger heap.
> Using the java DirectoryStream can avoid holding all file names in memory
> during file listing. Also, a multi-round directory cleaner can be introduced
> to delete files while disk quota is exceeded.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
