[
https://issues.apache.org/jira/browse/STORM-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051480#comment-15051480
]
ASF GitHub Bot commented on STORM-901:
--------------------------------------
Github user erikdw commented on the pull request:
https://github.com/apache/storm/pull/822#issuecomment-163721659
@revans2 : good memory, my use-case is indeed storm-on-mesos. And in the
way my company deploys it, we leverage the `storm.log.dir` to force the logs to
be written into a known location where we can forward them to various
downstream log-indexing/storing services. That is the use case we want to be
supported *somehow*. I'm not particularly married to `storm.log.dir`, so your
proposal to have the `worker-artifacts` go to some other location seems
sufficient for my use case. I'll explicitly state that I would expect it to be
a knob that is configurable from within `storm.yaml`.
BTW, changing to a hierarchical layout from a flat style was concerning at
first blush, but on further reflection I believe it will work just fine. Our
tools for log-forwarding can use globs/wildcards so it should be ok.
> Worker Artifacts Directory
> --------------------------
>
> Key: STORM-901
> URL: https://issues.apache.org/jira/browse/STORM-901
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Zhuo Liu
> Fix For: 0.11.0
>
> Attachments: daemon-logs-view.png, worker-logs-view.png
>
>
> At Yahoo we have added in one separate directory that not only stores each
> topology's every worker's log files, but also provides a place where the
> worker can place other files which helps with debugging (gc logs and heap
> dump files). It is a new directory that soft-links to the current working
> directory of the worker process. The oldest files in each worker's artifacts
> directory are cleaned up if they get too large (exceeding the per-dir quota
> or the global quota).
> This work is already done, but needs to be put back into open source.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
