Which version of Fluo are you using? The properties 'io.fluo.log.*' were used before Apache incubation in 1.0.0-beta-1 or 1.0.0-beta-2. These properties were renamed in Fluo 1.0.0 to 'fluo.log.*'. You could be having problems if you built your application using Fluo 1.0.0-beta-2 and ran it using a Fluo 1.0.0 install.
On Tue, Nov 1, 2016 at 4:45 PM Meier, Caleb <[email protected]> wrote: > Hey Keith, > > Not seeing the worker logs in my container. Think my logback is > configured incorrectly... > > [root@c190sv193 container_1476563020088_0069_01_000002]# grep Rolling > stdout > 18:36:49,174 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - > About to instantiate appender of type > [ch.qos.logback.core.rolling.RollingFileAppender] > 18:36:49,215 |-INFO in > ch.qos.logback.core.rolling.FixedWindowRollingPolicy@5ab860f4 - No > compression will be used > 18:36:49,230 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file > name: > io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log.host_IS_UNDEFINED.log > 18:36:49,230 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is > set to > [io.fluo.log.dir_IS_UNDEFINED/io.fluo.log.app_IS_UNDEFINED_io.fluo.log.host_IS_UNDEFINED.log] > > -----Original Message----- > From: Keith Turner [mailto:[email protected]] > Sent: Tuesday, November 01, 2016 4:33 PM > To: [email protected] > Subject: Re: debugging fluo > > Caleb, > > I just ran Fluo locally with Uno. Below is some info I am seeing. > Do you see anything about RollingFileAppender in the stdout file? > > $ pwd > > /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004 > $ ls > stderr stdout worker_1_host1.log > $ grep Rolling stdout > 16:17:31,557 |-INFO in ch.qos.logback.core.joran.action.AppenderAction > - About to instantiate appender of type > [ch.qos.logback.core.rolling.RollingFileAppender] > 16:17:31,574 |-INFO in > ch.qos.logback.core.rolling.FixedWindowRollingPolicy@bc1b008 - No > compression will be used > 16:17:31,583 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file > name: > /home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004/worker_1_host1 > 16:17:31,583 |-INFO in > ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is > set to > [/home/kturner/uno/install/logs/yarn/application_1478030941494_0001/container_1478030941494_0001_01_000004/worker_1_host1] > > Keith > > On Tue, Nov 1, 2016 at 4:22 PM, Meier, Caleb <[email protected]> > wrote: > > Hey Mike, > > > > So I’m not seeing any worker logs on my machines. The search > > > > find / | grep worker > > > > yielded nothing. Any ideas as to why these don't exist? I'm not sure > > that my fluo.log.dir system property is successfully being set by the > LogbackUtil class. > > Maybe this has something to do with it? > > > > > > > > > > > > -----Original Message----- > > From: Mike Walch [mailto:[email protected]] > > Sent: Tuesday, November 01, 2016 4:03 PM > > To: [email protected] > > Subject: Re: debugging fluo > > > > Were you able to find a worker_*.log file for each of your workers? > > > > Below are some tips for debugging: > > > > - Each YARN container should have a 'stdout' and 'stderr' file. These > files may have helpful error messages. Especially if a worker failed to > start. Also, any calls to System.out and System.err in your observer will > be printed to these files. > > - When running Fluo in YARN, Fluo must use Logback for logging (due to a > hard requirment by Twill). Logback is configured using > /path/to/fluo/conf/logback.xml. You should review this configuration but > the root logger is configured by default to print any message that is the > debug level or higher. > > - If you configured multiple workers, each worker will run in a > different container and have a different worker_*.log file. > > - When a worker starts up, it prints its configuration to worker_*.log. > > Make sure that you configured your observers using the property > 'fluo.observer.*' > > > > -Mike > > > > On Tue, Nov 1, 2016 at 3:33 PM Meier, Caleb <[email protected]> > wrote: > > > >> Do you have any tips for how to make Observers log to the log files > >> found in the directory specified by 'yarn.nodemanager.log-dirs'? > >> > >> -----Original Message----- > >> From: Mike Walch [mailto:[email protected]] > >> Sent: Tuesday, November 01, 2016 2:36 PM > >> To: [email protected] > >> Subject: Re: debugging fluo > >> > >> Hi Caleb, > >> > >> The logs for a Fluo application can be found in YARN but they are > >> tricky to find. Fluo should have better documentation on this which I > will add now. > >> > >> The easiest way to view the logs for a Fluo application is to use the > >> web interface for the YARN resource manager ( > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8088_ > >> c > >> luster&d=CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdz > >> Y > >> C2kksVZR5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk > >> 6 mq5Tns5Aewd2s&s=_4PS5z_vu1bkhpZBdzJXjbGsCvMBboMqoLBIOBnRAEY&e= > >> ). > >> First, click on the application ID (i.e application_*) of your Fluo > >> application and then click on the latest attempt ID (appattempt_*). > >> You should see a list of containers. There should be a container for > >> the application master (typically container 1), a Fluo oracle > >> (typically container 2), and Fluo workers (containers 3+). You can > >> view the log files produced by a container by clicking on its 'logs' > >> link. Logs from Fluo observers will be in the worker_*.log file for > >> each of your worker containers. > >> > >> If you don't want to use the YARN resource manager web interface, you > >> can also view these logs in the directory specified by > >> 'yarn.nodemanager.log-dirs' of your 'yarn-site.xml' config. This > >> method works well on one machine but on cluster your containers will > >> probably be on different machines. See the YARN documentation below > >> for more info about this property: > >> > >> > >> https://urldefense.proofpoint.com/v2/url?u=https-3A__hadoop.apache.or > >> g > >> _docs_r2.7.0_hadoop-2Dyarn_hadoop-2Dyarn-2Dcommon_yarn-2Ddefault.xml& > >> d > >> =CwIBaQ&c=Nwf-pp4xtYRe0sCRVM8_LWH54joYF7EKmrYIdfxIq10&r=vuVdzYC2kksVZ > >> R > >> 5STiFwDpzJ7CrMHCgeo_4WXTD0qo8&m=S88pZ1xYAkVw1LehCcB3YDzFFeEKk6mq5Tns5 > >> A ewd2s&s=SS8YTOgIAWUmBnKkHN2Eu2-h6WyEHqlNvJO-D5EKFiI&e= > >> > >> Best, > >> Mike > >> > >> On Tue, Nov 1, 2016 at 1:29 PM Meier, Caleb <[email protected]> > >> wrote: > >> > >> Hello, > >> > >> I'm attempting to debug a Fluo application and am having difficulty > >> locating the logs for my observers. I've looked within the logs for > >> hadoop-yarn, but am not seeing any logging statements for my observers. > >> Where do observers log out of the box in a normal cloudera distribution? > >> Do I need to do something else in addition to logging to get my > >> observers to generate logs? > >> > >> Thanks, > >> Caleb > >> >
