[
https://issues.apache.org/jira/browse/STORM-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051809#comment-15051809
]
ASF GitHub Bot commented on STORM-901:
--------------------------------------
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/822#issuecomment-163780366
It would be a rather large change to support two different directory
structures, one for logs and one for everything else. It is not impossible to
do, but it would add quite a bit of complexity to something that is already
more complex then I like.
I guess it comes down to how often do you and your customers access the
logs through the storm UI or some other automated log system vs how often you
access the logs by going to the box directly and reading them directly. For us
our clusters have grown large enough that very few people try to read the logs
directly. Finding the right node is such a pain that the added directly
structure is not something they have complained about.
Would having the worker-artifacts directory structure by default be under
`storm.log.dir` but also have a separate config be an acceptable solution?
This is a very simple change to make, vs trying to split the data.
> Worker Artifacts Directory
> --------------------------
>
> Key: STORM-901
> URL: https://issues.apache.org/jira/browse/STORM-901
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Zhuo Liu
> Fix For: 0.11.0
>
> Attachments: daemon-logs-view.png, worker-logs-view.png
>
>
> At Yahoo we have added in one separate directory that not only stores each
> topology's every worker's log files, but also provides a place where the
> worker can place other files which helps with debugging (gc logs and heap
> dump files). It is a new directory that soft-links to the current working
> directory of the worker process. The oldest files in each worker's artifacts
> directory are cleaned up if they get too large (exceeding the per-dir quota
> or the global quota).
> This work is already done, but needs to be put back into open source.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
