Hi folks, Hope you are all doing well.
My company is using Livy to submit spark jobs on AWS. On one project where we are submitting the jobs to compact data at about one job per minute per Jenkins job, we are getting 40x errors from Livy. There are about 20 such Jenkins tasks running at the time. When we restart the Jenkins task, it continues to submit jobs successfully. I'm wondering if there is a rate limit in the default configuration of Livy. Is this the case? Could you please point me to where this can be updated. Thank you, Sadha