I think you have plenty of ways to get application log, either via command line or programmatically, even with yarn RM UI. Since there already has several ways to get application log, so currently we don't have a plan to address this.
2017-11-27 20:04 GMT+08:00 Partridge, Lucas (GE Aviation) < lucas.partri...@ge.com>: > Thanks Jerry. > > > > “Currently there's no Livy api for you to get application log via REST > API.” > > - That’s a real shame. Are there any plans to address this? Because this > really limits the usefulness of Livy when I’m using Yarn cluster mode. It > seems I have to make a choice between running my client in a scalable > manner (yarn cluster) or being able to see the logs programmatically (yarn > client), but not both. My client is multi-threaded; I don’t want it to > have to host multiple concurrent Spark driver applications but it looks > like I might have no choice about this. > > > > Thanks, Lucas. > > > > *From:* Saisai Shao [mailto:sai.sai.s...@gmail.com] > *Sent:* 27 November 2017 02:19 > *To:* user@livy.incubator.apache.org > *Subject:* EXT: Re: How to access Python script's stdout when running in > Yarn cluster mode? > > > > Since you're running with yarn cluster mode, the output from your python > script should be part of your yarn application log. you can get it via yarn > command like yarn log -applicationId <applictionId>, or others like Yarn > UI. Currently there's no Livy api for you to get application log via REST > API. > > > > Thanks > > Jerry > > > > 2017-11-24 20:27 GMT+08:00 Partridge, Lucas (GE Aviation) < > lucas.partri...@ge.com>: > > Hi, > > > > I’m using Livy’s GET /batches/{batchId}/log method to fetch the log lines > from a Python script I’m running on Spark in Yarn cluster mode. > Unfortunately the stdout from my Python script is not included in the log > lines returned by GET /batches/{batchId}/log! > > > > Is this by design, or an unfortunate by-product of running in Yarn cluster > mode? > > > > If this is intentional does anyone know how I can access the stdout from > my Python script via Livy please? Preferably without having to change my > REST client (a Java app) to use Yarn’s client deployment mode. > > > > Thanks, > > Lucas. > > > > >
