Since you're running with yarn cluster mode, the output from your python
script should be part of your yarn application log. you can get it via yarn
command like yarn log -applicationId <applictionId>, or others like Yarn
UI. Currently there's no Livy api for you to get application log via REST
API.
Thanks
Jerry
2017-11-24 20:27 GMT+08:00 Partridge, Lucas (GE Aviation) <
lucas.partri...@ge.com>:
> Hi,
>
>
>
> I’m using Livy’s GET /batches/{batchId}/log method to fetch the log lines
> from a Python script I’m running on Spark in Yarn cluster mode.
> Unfortunately the stdout from my Python script is not included in the log
> lines returned by GET /batches/{batchId}/log!
>
>
>
> Is this by design, or an unfortunate by-product of running in Yarn cluster
> mode?
>
>
>
> If this is intentional does anyone know how I can access the stdout from
> my Python script via Livy please? Preferably without having to change my
> REST client (a Java app) to use Yarn’s client deployment mode.
>
>
>
> Thanks,
>
> Lucas.
>
>
>
