Re: Unable to launch job with 100 SQL queries in yarn cluster

2018-04-16 Thread Fabian Hueske
Hi Adrian, Thanks reaching out to the community. I don't think that this is an issue with Flink's SQL support. SQL queries are translated into regular streaming (or batch) jobs. The JM might just be overloaded by too many jobs. Since you are running in a YARN environment, it might make sense to

Unable to launch job with 100 SQL queries in yarn cluster

2018-04-13 Thread Adrian Hains
Hi, We are having trouble scaling up Flink to execute a collection of SQL queries on a yarn cluster. Has anyone run this kind of workload on a cluster? Any tips on how to get past this issue? With a high number of Flink SQL queries (100 instances of the query at the bottom of this message), the