I have a Livy application that I'm trying to tune as I'm seeing some performance issue when the queries are fast queries. I've wrapped my queries with a timer that logs the time taken. The spark code executed typically takes 50ms to 150ms. I'm querying Livy every 500ms looking for my response, and generally it doesn't succeed until the third check. It seems Livy itself is spending up to an extra 1000ms. Where is Livy spending this time? Are there any tuning parameters I can adjust?
Also, I am having difficulty changing any of the settings in livy-client.conf. I placed the file in /etc/hadoop/conf and livy/conf folder but my settings seem to get ignored. Thanks Tim
