Thank you all for the tips. Will give a try. On Fri, Apr 13, 2018 at 12:13 PM, Gary Yao <g...@data-artisans.com> wrote:
> Hi, > > I see two options: > > 1. You can login to the slave machines, which run the NodeManagers, and > access > the container logs. The path of the container logs can be configured in > yarn-site.xml with the key yarn.nodemanager.log-dirs. In my tests with > EMR, the > logs are stored at /var/log/hadoop-yarn/containers/ by default. For > example: > > sudo ls /var/log/hadoop-yarn/containers/application_ > 1523645835493_0002/container_1523645835493_0002_01_000002 > taskmanager.err taskmanager.log taskmanager.out > > 2. You can download the logs via HTTP from Flink: > > http://host:port/jobmanager/log > http://host:port/taskmanagers/<taskmanager-id>/log > > To get a list of taskmanagers: > > http://host:port/taskmanagers > > Best, > Gary > > On Fri, Apr 13, 2018 at 2:09 AM, Tao Xia <t...@udacity.com> wrote: > >> Any good way to get access container logs from a running Flink app in >> YARN cluster in EMR? >> You can view the logs through YARN UI. But cannot programmatically access >> it and send to other services. >> The log aggregator only runs when the application finishes or a minimum >> 3600 secs copy. Any way we can get the logs more frequently? >> > >
