On 25/06/2018 22:29, Meisam Fathi wrote:
> If you are using YARN the driver logs are always available at YARN
> resource manager.
YARN keeps the driver logs when you run in yarn-cluster mode, I'm more
interested in yarn-client and spark local modes.
> You can also see the logs from /sessions/{sessionid}/logs
My main question is whether Livy persists the logs somewhere on the
filesystem or they are simply gone when the server is restarted ?
>
> Thanks,
> Meisam
>
> On Sun, Jun 24, 2018 at 1:37 PM Abbass > wrote:
>
> Guys any idea about where Livy keeps job execution logs (Spark driver
> logs) ?
>
> I saw a couple of lines here :
>
> https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/server/batch/BatchSession.scala#L90
>
>
> to redirect stdout & stderr somewhere.
>
> Is there anyway to persist these logs on the filesystem ?
>
> Thanks,
>
> As a recipient of an email from Talend, your contact personal data
> will be on our systems. Please see our contacts privacy notice at
> Talend, Inc.
>
>
