[
https://issues.apache.org/jira/browse/STORM-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562565#comment-16562565
]
Jorge Moraleda edited comment on STORM-1759 at 7/30/18 9:55 PM:
----------------------------------------------------------------
I think routing all log viewing through a single host/port running storm ui as
requested in https://issues.apache.org/jira/browse/STORM-2708 is the most
desirable approach for log viewiing in a container based deployment.
The obvious alternative is leaving storm logging as is. This forces one to
either give up the convenience of accessing all logs from the browsers from the
ui and instead requiring to ssh into each container, or an impractically
complex setup that dynamically maps and open each supervisor's logviewer port
to a port accessible from outside the cluster and handles container restarts
and dynamic scaling of supervisors.
This issue has in some form been going on for almost 3 years in some form
https://issues.apache.org/jira/browse/STORM-1342, but I believe it is now much
more urgent than it used to be since deployment in containers is a lot more
common now. Moreover I believe the solution should be orchestration agnostic.
In particular it should not rely on runnning under mesos. It also should also
not require running additional services: Since both a zookeeper and a nimbus
are already required for any deployment, that should be sufficient for
coordination for storm ui to display the logs of individual supervisors at its
own port.
was (Author: jmoraleda):
I think routing all log viewing through a single host/port running storm ui as
requested in [#2708] is the most desirable approach for log viewiing in a
container based deployment.
The obvious alternative is leaving storm logging as is. This forces one to
either give up the convenience of accessing all logs from the browsers from the
ui and instead requiring to ssh into each container, or an impractically
complex setup that dynamically maps and open each supervisor's logviewer port
to a port accessible from outside the cluster and handles container restarts
and dynamic scaling of supervisors.
This issue has in some form been going on for almost 3 years in some form
[#1342], but I believe it is now much more urgent than it used to be since
deployment in containers is a lot more common now. Moreover I believe the
solution should be orchestration agnostic. In particular it should not rely on
runnning under mesos. It also should also not require running additional
services: Since both a zookeeper and a nimbus are already required for any
deployment, that should be sufficient for coordination for storm ui to display
the logs of individual supervisors at its own port.
> Viewing logs from the Storm UI doesn't work in dockerized environment
> ---------------------------------------------------------------------
>
> Key: STORM-1759
> URL: https://issues.apache.org/jira/browse/STORM-1759
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-ui
> Affects Versions: 0.10.0, 1.0.0, 0.9.6
> Reporter: Elisey Zanko
> Priority: Major
>
> I run the Storm using the following docker-compose.yml
> {code}
> version: '2'
> services:
> zookeeper:
> image: jplock/zookeeper:3.4.8
> restart: always
> nimbus:
> image: 31z4/storm:1.0.0
> command: nimbus -c storm.log.dir="/logs" -c
> storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
> depends_on:
> - zookeeper
> restart: always
> ports:
> - 6627:6627
> volumes:
> - logs:/logs
> supervisor:
> image: 31z4/storm:1.0.0
> command: supervisor -c storm.log.dir="/logs" -c
> storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
> depends_on:
> - nimbus
> restart: always
> volumes:
> - logs:/logs
> logviewer:
> image: 31z4/storm:1.0.0
> command: logviewer -c storm.log.dir="/logs"
> restart: always
> ports:
> - 8000:8000
> volumes:
> - logs:/logs
> ui:
> image: 31z4/storm:1.0.0
> command: ui -c storm.log.dir="/logs" -c nimbus.host="nimbus"
> depends_on:
> - nimbus
> - logviewer
> restart: always
> ports:
> - 8080:8080
> volumes:
> - logs:/log
> volumes:
> logs: {}
> {code}
> And opening the logs from the Storm UI doesn't work because all links are
> pointing to different container ids as hosts.
> I guess adding an ability to explicitly specify the logviewer host in the
> storm.yaml would solve the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
