Hi,
Seem the QPS result is not ideal in your cluster. As far as I know, QPS of
Kylin in production cluster often reach to 40 or higher. The reason maybe
caused by huge data size or cube is not deigned well. You may check
http://kylin.apache.org/blog/2016/02/18/new-aggregation-group/ or try tuning
parameter.
On the another hand, I think you may try to use some well-known benchmark
dataset and query to do a more convincing benchmark. You may check
https://github.com/Kyligence/ssb-kylin and
https://github.com/Kyligence/kylin-tpch.
-----------------
-----------------
Best wishes to you !
From ：Xiaoxiang Yu
At 2019-07-11 01:35:23, "Jayakumar Balasubramanya" <[email protected]>
wrote:
Hi,
We are testing out our newly setup production infrastructure for
scalability in terms of Query response. We are using a AWS EMR cluster with
Kylin installed on the master node of the EMR. We have a 4 node cluster with
each node having a 4 core, 16 GB ( will be 32 GB on go-live) configuration.
While Kylin should be able to easily scale up, we seem to be hitting some choke
point. When we fire 40 queries at the same point in time, all the queries seem
to be stuck for about 6 seconds before we get any response. On a single query
being fired, we are getting sub-second response. Any pointers to what we should
be checking and any findings of a similar exercise will help a lot. We are
using Gatling for building and running the tests.
When 40 queries are executed on kylin parallelly (at once) the
response time distribution is as below, with min, max and mean response time as
6557,7580 and 6887 ms respectively.
Kylin doesn’t seem to handle parallel execution of queries.
But when 40 queries are executed over 40 seconds i.e one query
executed per second the response time distribution is as below, with min, max
and mean response time as 110,2820 and 248 ms respectively.
This seems fine as the mean response time in less than a
second.
