jerryshao commented on a change in pull request #242: [LIVY-336] Livy should not spawn one thread per job to track the job on Yarn URL: https://github.com/apache/incubator-livy/pull/242#discussion_r333315065
########## File path: conf/livy.conf.template ########## @@ -121,15 +121,21 @@ # If Livy can't find the yarn app within this time, consider it lost. # livy.server.yarn.app-lookup-timeout = 120s +# How long to find the yarn app +# livy.server.yarn.app-lookup-interval = 1s # When the cluster is busy, we may fail to launch yarn app in app-lookup-timeout, then it would # cause session leakage, so we need to check session leakage. # How long to check livy session leakage # livy.server.yarn.app-leakage.check-timeout = 600s # how often to check livy session leakage # livy.server.yarn.app-leakage.check-interval = 60s +# how long to monitor the yarn app +# livy.server.yarn.app-monitor.timeout = 10000ms +# If Livy can't monitor the yarn app successfully within this max times, consider the app failed. +# livy.server.yarn.app-monitor.max-failed.times = 120 # How often Livy polls YARN to refresh YARN app state. -# livy.server.yarn.poll-interval = 5s +# livy.server.yarn.poll-interval = 500ms Review comment: I'm afraid this will bring overhead to YARN if it is a large cluster. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
