Hey Lukas, Could you paste the logs for both the run-job.sh and the .log file that's being produced by the container? I see no mention of "local-process-container" in either 0.7.0 or 0.8.0. By default, the ShellCommandBuilder should set SAMZA_CONTAINER_NAME to your job.name, and run-class.sh should set samza-container.name to SAMZA_CONTAINER_NAME.
Cheers, Chris On 12/23/14 2:24 PM, "Lukas Steiblys" <[email protected]> wrote: >Thanks! I made the deploy script switch to the package root directory >before >running the job. The only problem now is that the logs are written to a >local-process-container.log file instead of JOB-NAME.log file what is >specified in the SAMZA_CONTAINER_NAME environment variable. > >Lukas > >-----Original Message----- >From: Chris Riccomini >Sent: Tuesday, December 23, 2014 1:58 PM >To: [email protected] >Subject: Re: Changes to logging in Samza 0.8 > >Hey Lukas, > >It looks like you are starting run-job.sh from outside the package root >(what you get when you un-tar your package tarball). By default, the >ProcessJob (via ShellCommandBuilder) uses this: > > def getCommand = >getOption(ShellCommandConfig.COMMAND_SHELL_EXECUTE).getOrElse("bin/run-con >t >ainer.sh") > > >If this is run from outside the package root, you'll get the exception you >see. I think this might work: > > task.execute=./deploy/samza/bin/run-container.sh > >To specify the location of the run-container.sh script. > > >Note: as expected, the logs show that the run-job.sh script is picking up >-Dlog4j.configuration=file:./deploy/samza/bin/log4j-console.xml. When the >ProcssJob works, I'd expect that process will pick up the lib/log4j.xml. >I've never run the run-job.sh script with ProcessJob from outside of the >package root, though. If it doesn't work, please post issues, so we can >open the appropriate JIRAs. > >Cheers, >Chris > >On 12/23/14 1:49 PM, "Lukas Steiblys" <[email protected]> wrote: > >>Here's the log: http://paste.ofcode.org/HLCvT2j8BY6nLhpqQ6Ld3b >> >>Lukas >> >>-----Original Message----- >>From: Chris Riccomini >>Sent: Tuesday, December 23, 2014 1:22 PM >>To: [email protected] >>Subject: Re: Changes to logging in Samza 0.8 >> >>Hey Lukas, >> >>Log attachments seem to be filtered out. Could you try posting on a >>public >>paste, or github gist? >> >>Cheers, >>Chris >> >>On 12/23/14 1:20 PM, "Lukas Steiblys" <[email protected]> wrote: >> >>>Unfortunately, that didn't help. Not only did the log show up in STDOUT, >>>the >>>job also failed to start (but the process didn't stop). Log attached. >>> >>>Lukas >>> >>>-----Original Message----- >>>From: Chris Riccomini >>>Sent: Tuesday, December 23, 2014 1:08 PM >>>To: [email protected] >>>Subject: Re: Changes to logging in Samza 0.8 >>> >>>Hey Lukas, >>> >>>I believe this is because you're using: >>> >>> job.factory.class=org.apache.samza.job.local.ThreadJobFactory >>> >>>Config settings you have that need to be set at JVM start time can't be >>>applied using the ThreadJobFactory, since the JVM has already started. >>>As >>>a result, you get whatever JVM settings your run-job.sh script uses. For >>>log4j, I believe this means it'll pick up the log4j-console.xml in your >>>bin directory. >>> >>>Can you try using: >>> >>> job.factory.class=org.apache.samza.job.local.ProcessJobFactory >>> >>> >>>Cheers, >>>Chris >>> >>>On 12/23/14 1:00 PM, "Lukas Steiblys" <[email protected]> wrote: >>> >>>>I do not have a custom task.opts. >>>> >>>>Here's the full package we deploy: >>>>http://imbusy.org/temp/samza-package-0.1-SNAPSHOT-dist.tar.gz . I have >>>>also >>>>attached one of the deploy scripts we use for one of the five jobs >>>>available. They are all run locally. >>>> >>>>Lukas >>>> >>>>-----Original Message----- >>>>From: Chris Riccomini >>>>Sent: Tuesday, December 23, 2014 12:32 PM >>>>To: [email protected] >>>>Subject: Re: Changes to logging in Samza 0.8 >>>> >>>>Hey Lukas, >>>> >>>>The changes are probably from this ticket: >>>> >>>> https://issues.apache.org/jira/browse/SAMZA-109 >>>> >>>>The behavior you're observing does not sound correct, though. By >>>>default, >>>>if you have a log4j.xml in your lib directory, and don't have a custom >>>>task.opts, then you should get proper .log files. Do you have a custom >>>>task.opts? If so, could you paste it? >>>> >>>>Cheers, >>>>Chris >>>> >>>>On 12/23/14 11:23 AM, "Lukas Steiblys" <[email protected]> wrote: >>>> >>>>>I have recently upgraded from Samza 0.7 to 0.8 and noticed that, >>>>>instead >>>>>of logging to a file using log4j to the log directory specified in the >>>>>environment variable SAMZA_LOG_DIR, all the logs are dumped to STDOUT. >>>>> >>>>>What changed in 0.8 and what¹s the path to upgrading to get the old >>>>>functionality back? >>>>> >>>>>Lukas >> >
