Hi,
We are testing out our newly setup production infrastructure for
scalability in terms of Query response. We are using a AWS EMR cluster with
Kylin installed on the master node of the EMR. We have a 4 node cluster with
each node having a 4 core, 16 GB ( will be 32 GB on go-live) configuration.
While Kylin should be able to easily scale up, we seem to be hitting some choke
point. When we fire 40 queries at the same point in time, all the queries seem
to be stuck for about 6 seconds before we get any response. On a single query
being fired, we are getting sub-second response. Any pointers to what we should
be checking and any findings of a similar exercise will help a lot. We are
using Gatling for building and running the tests.
When 40 queries are executed on kylin parallelly (at once) the
response time distribution is as below, with min, max and mean response time as
6557,7580 and 6887 ms respectively.
Kylin doesn't seem to handle parallel execution of queries.
[cid:[email protected]][cid:[email protected]]
But when 40 queries are executed over 40 seconds i.e one query
executed per second the response time distribution is as below, with min, max
and mean response time as 110,2820 and 248 ms respectively.
This seems fine as the mean response time in less than a
second.
[cid:[email protected]][cid:[email protected]]
