Hello
I am new to Livy and still exploring but looking for ways to get driver
logs in cluster mode. Currently, we have bulky edge nodes which are used to
submit spark applications on YARN. We use client mode instead of
recommended cluster mode due to easy access of realtime driver side logs.
Livy would help us to remove need of bulky edge nodes instead submit spark
apps from anywhere in cluster mode. The only problem I am facing is not
having an access to driver side logs.
I would appreciate if someone help me understand if there is a way to get
driver side logs in Livy. It will be interesting to learn how other
people/companies use Livy + Spark + logging services too.
Thanks
Swapnil
