I suppose you don't see oracle files either? $ pwd /home/kturner/uno/install/logs/yarn/application_1478030941494_0001 $ find . . ./container_1478030941494_0001_01_000001 ./container_1478030941494_0001_01_000001/stdout ./container_1478030941494_0001_01_000001/stderr ./container_1478030941494_0001_01_000002 ./container_1478030941494_0001_01_000002/stdout ./container_1478030941494_0001_01_000002/stderr ./container_1478030941494_0001_01_000002/oracle_0_host1 ./container_1478030941494_0001_01_000003 ./container_1478030941494_0001_01_000003/stdout ./container_1478030941494_0001_01_000003/stderr ./container_1478030941494_0001_01_000003/worker_0_host1 ./container_1478030941494_0001_01_000004 ./container_1478030941494_0001_01_000004/stdout ./container_1478030941494_0001_01_000004/stderr ./container_1478030941494_0001_01_000004/worker_1_host1
On Tue, Nov 1, 2016 at 4:40 PM, Meier, Caleb <caleb.me...@parsons.com> wrote: > Hey Keith, > > Not seeing the worker logs in my container. Think my logback is configured > incorrectly... > > [root@c190sv193 container_1476563020088_0069_01_000002]# grep Rolling stdout > 18:36:49,174 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - > About to instantiate appender of type > [ch.qos.logback.core.rolling.RollingFileAppender] > 18:36:49,215 |-INFO in > ch.qos.logback.core.rolling.FixedWindowRollingPolicy@5ab860f4 - No > compression will be used > 18:36:49,230 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] > - Active log file name: > io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log.host_IS_UNDEFINED.log > 18:36:49,230 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] > - File property is set to > [io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log.host_IS_UNDEFINED.log] > > -----Original Message----- > From: Keith Turner [mailto:ke...@deenlo.com] > Sent: Tuesday, November 01, 2016 4:33 PM > To: dev@fluo.incubator.apache.org > Subject: Re: debugging fluo > > Caleb, > > I just ran Fluo locally with Uno. Below is some info I am seeing. > Do you see anything about RollingFileAppender in the stdout file? > > $ pwd > /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004 > $ ls > stderr stdout worker_1_host1.log > $ grep Rolling stdout > 16:17:31,557 |-INFO in ch.qos.logback.core.joran.action.AppenderAction > - About to instantiate appender of type > [ch.qos.logback.core.rolling.RollingFileAppender] > 16:17:31,574 |-INFO in > ch.qos.logback.core.rolling.FixedWindowRollingPolicy@bc1b008 - No compression > will be used > 16:17:31,583 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: > /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004/worker_1_host1 > 16:17:31,583 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set > to > [/home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004/worker_1_host1] > > Keith > > On Tue, Nov 1, 2016 at 4:22 PM, Meier, Caleb <caleb.me...@parsons.com> wrote: >> Hey Mike, >> >> So I’m not seeing any worker logs on my machines. The search >> >> find / | grep worker >> >> yielded nothing. Any ideas as to why these don't exist? I'm not sure >> that my fluo.log.dir system property is successfully being set by the >> LogbackUtil class. >> Maybe this has something to do with it? >> >> >> >> >> >> -----Original Message----- >> From: Mike Walch [mailto:mwa...@apache.org] >> Sent: Tuesday, November 01, 2016 4:03 PM >> To: dev@fluo.incubator.apache.org >> Subject: Re: debugging fluo >> >> Were you able to find a worker_*.log file for each of your workers? >> >> Below are some tips for debugging: >> >> - Each YARN container should have a 'stdout' and 'stderr' file. These files >> may have helpful error messages. Especially if a worker failed to start. >> Also, any calls to System.out and System.err in your observer will be >> printed to these files. >> - When running Fluo in YARN, Fluo must use Logback for logging (due to a >> hard requirment by Twill). Logback is configured using >> /path/to/fluo/conf/logback.xml. You should review this configuration but >> the root logger is configured by default to print any message that is the >> debug level or higher. >> - If you configured multiple workers, each worker will run in a different >> container and have a different worker_*.log file. >> - When a worker starts up, it prints its configuration to worker_*.log. >> Make sure that you configured your observers using the property >> 'fluo.observer.*' >> >> -Mike >> >> On Tue, Nov 1, 2016 at 3:33 PM Meier, Caleb <caleb.me...@parsons.com> wrote: >> >>> Do you have any tips for how to make Observers log to the log files >>> found in the directory specified by 'yarn.nodemanager.log-dirs'? >>> >>> -----Original Message----- >>> From: Mike Walch [mailto:mwa...@apache.org] >>> Sent: Tuesday, November 01, 2016 2:36 PM >>> To: dev@fluo.incubator.apache.org >>> Subject: Re: debugging fluo >>> >>> Hi Caleb, >>> >>> The logs for a Fluo application can be found in YARN but they are >>> tricky to find. Fluo should have better documentation on this which I will >>> add now. >>> >>> The easiest way to view the logs for a Fluo application is to use the >>> web interface for the YARN resource manager ( >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8088_ >>> c >>> luster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdz >>> Y >>> C2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk >>> 6 mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e= >>> ). >>> First, click on the application ID (i.e application_*) of your Fluo >>> application and then click on the latest attempt ID (appattempt_*). >>> You should see a list of containers. There should be a container for >>> the application master (typically container 1), a Fluo oracle >>> (typically container 2), and Fluo workers (containers 3+). You can >>> view the log files produced by a container by clicking on its 'logs' >>> link. Logs from Fluo observers will be in the worker_*.log file for >>> each of your worker containers. >>> >>> If you don't want to use the YARN resource manager web interface, you >>> can also view these logs in the directory specified by >>> 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config. This >>> method works well on one machine but on cluster your containers will >>> probably be on different machines. See the YARN documentation below >>> for more info about this property: >>> >>> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.or >>> g >>> _docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xml& >>> d >>> =CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZ >>> R >>> 5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5 >>> A ewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e= >>> >>> Best, >>> Mike >>> >>> On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <caleb.me...@parsons.com> >>> wrote: >>> >>> Hello, >>> >>> I'm attempting to debug a Fluo application and am having difficulty >>> locating the logs for my observers. I've looked within the logs for >>> hadoop-yarn, but am not seeing any logging statements for my observers. >>> Where do observers log out of the box in a normal cloudera distribution? >>> Do I need to do something else in addition to logging to get my >>> observers to generate logs? >>> >>> Thanks, >>> Caleb >>>
