I propose the following configurations to support this feature: # How long to retain an inactive interactive session before cleaning it up livy.server.session.timeout
# How long to retain an interactive session that ran successfully livy.server.session.success.retaintion # How long to retain an interactive session that did NOT run successfully livy.server.session.failure.retaintion # How long to retain an batch session that ran successfully livy.server.batch.success.retaintion # How long to retain an batch session that did NOT ran successfully livy.server.batch.failure.retaintion We added a variation of these configs to Livy here at PayPal, which helps us manage our Spark jobs with more control. Particularly, these configs help us clean up interactive sessions created on Notebooks more efficiently. They also help us clean up failed jobs quicker. If there is an interest, I can send a PR. On Tue, Dec 11, 2018 at 4:11 AM Praveen Muthusamy <praveen....@gmail.com> wrote: > Hi, > > Is there any setting in livy server that we can keep, so that the batch > session info will be retained longer ? When a batch application is > submitted and gets successful, that session is cleared, there is no way for > a external program to get the status of this batch session post that. > > Regards, > Praveen M >
