Hi, please for questions related to how to use Livy, use only the users list and not the dev one. Depending on your needs, there may be different answers to your question. First, I'd recommend you to check the documentation of Livy's REST API, you might find what you need there: - https://livy.incubator.apache.org/docs/latest/rest-api.html#get-sessionssessionidlog - https://livy.incubator.apache.org/docs/latest/rest-api.html#get-batchesbatchidlog Second, you can always read the logs using YARN, since the Spark application is executed on YARN. Thanks, Marco
Il giorno gio 7 mar 2019 alle ore 16:57 Swapnil Shinde < swapnilushi...@gmail.com> ha scritto: > Hello > I am new to Livy and still exploring but looking for ways to get driver > logs in cluster mode. Currently, we have bulky edge nodes which are used to > submit spark applications on YARN. We use client mode instead of > recommended cluster mode due to easy access of realtime driver side logs. > Livy would help us to remove need of bulky edge nodes instead submit spark > apps from anywhere in cluster mode. The only problem I am facing is not > having an access to driver side logs. > I would appreciate if someone help me understand if there is a way to get > driver side logs in Livy. It will be interesting to learn how other > people/companies use Livy + Spark + logging services too. > > Thanks > Swapnil >
