I see two options:
1. You can login to the slave machines, which run the NodeManagers, and
the container logs. The path of the container logs can be configured in
yarn-site.xml with the key yarn.nodemanager.log-dirs. In my tests with EMR,
logs are stored at /var/log/hadoop-yarn/containers/ by default. For example:
taskmanager.err taskmanager.log taskmanager.out
2. You can download the logs via HTTP from Flink:
To get a list of taskmanagers:
On Fri, Apr 13, 2018 at 2:09 AM, Tao Xia <t...@udacity.com> wrote:
> Any good way to get access container logs from a running Flink app in YARN
> cluster in EMR?
> You can view the logs through YARN UI. But cannot programmatically access
> it and send to other services.
> The log aggregator only runs when the application finishes or a minimum
> 3600 secs copy. Any way we can get the logs more frequently?