Do you have any tips for how to make Observers log to the log files found in the directory specified by 'yarn.nodemanager.log-dirs'?
-----Original Message----- From: Mike Walch [mailto:[email protected]] Sent: Tuesday, November 01, 2016 2:36 PM To: [email protected] Subject: Re: debugging fluo Hi Caleb, The logs for a Fluo application can be found in YARN but they are tricky to find. Fluo should have better documentation on this which I will add now. The easiest way to view the logs for a Fluo application is to use the web interface for the YARN resource manager (https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8088_cluster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e= ). First, click on the application ID (i.e application_*) of your Fluo application and then click on the latest attempt ID (appattempt_*). You should see a list of containers. There should be a container for the application master (typically container 1), a Fluo oracle (typically container 2), and Fluo workers (containers 3+). You can view the log files produced by a container by clicking on its 'logs' link. Logs from Fluo observers will be in the worker_*.log file for each of your worker containers. If you don't want to use the YARN resource manager web interface, you can also view these logs in the directory specified by 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config. This method works well on one machine but on cluster your containers will probably be on different machines. See the YARN documentation below for more info about this property: https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.org_docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xml&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5Aewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e= Best, Mike On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <[email protected]> wrote: Hello, I'm attempting to debug a Fluo application and am having difficulty locating the logs for my observers. I've looked within the logs for hadoop-yarn, but am not seeing any logging statements for my observers. Where do observers log out of the box in a normal cloudera distribution? Do I need to do something else in addition to logging to get my observers to generate logs? Thanks, Caleb
