Hi Chen, You can set the yarn.nodemanage.delete.debug-delay-sec config in yarn-site.xml, in order to retain the logs after the job crashes. Refer https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
(This was intended for debug purpose). If you need to retain log for longer period of time for purposes other than debugging, you should look into yarn's log aggregation features. Alternatively, you can use Samza's StreamAppender to produce the log data to a stream. You can then publish it to ELK for further lookup. It really depends on your requirements. Please note that the StreamAppender has a bug in the master branch that is pending resolution in https://issues.apache.org/jira/browse/SAMZA-723 Cheers! Navina On Mon, Aug 3, 2015 at 2:26 PM, Chen Song <chen.song...@gmail.com> wrote: > Dump question. > > When running samza job on YARN, I found that container logs gone after > days. Is there a YARN config to keep the logs from being deleted for > long-lived streaming job? > > -- > Chen Song > -- Navina R.
