[
https://issues.apache.org/jira/browse/KYLIN-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649744#comment-16649744
]
huaicui commented on KYLIN-3601:
--------------------------------
We use Gatling to loading test;
This is our test script:
import java.util.concurrent.TimeUnit
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
class KylinSimulation2 extends Simulation {
val httpConf = http
.acceptHeader("application/json")
.contentTypeHeader("application/json")
val headers_json = Map(
"Content-Type" -> "application/json",
"Authorization" -> "Basic QURNSU46S1lMSU4="
)
val scn2 = scenario("KylinServer6").during(60) { // 60 means keep sending
requests for about 60s
exec(
http("KylinQuery4")
.post("http://sap-datanode5:7070/kylin/api/query";)
.headers(headers_json)
.body(StringBody("{ \"sql\":\"select CLIENTAGENT, STARTDAY, COUNT(*)
TOTAL_NUMBER from XXRPT_HGSMEETINGUSERREPORT_VIEW where SITEID = ? group by
STARTDAY, CLIENTAGENT\", \"project\":\"CCA_REPORT\",
\"params\":[{\"className\":\"java.lang.String\",\"value\":\"1000182\"}]
}")).asJSON
)
}
setUp(
scn2.inject(atOnceUsers(150)) // 150 users(threads)
).protocols(httpConf)
}
We use 150 threads to loading test, kylin.query.statement-cache-max-num-per-key
is 200; We found the number of PrepareStatement instance would be more and more
like the attaches(CreatedCount will be larger and larger); No matter how many
concurrent that will trigger this issue;
> The max connection number generated by the PreparedContextPool is
> inconsistent with the configuration.
> ------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-3601
> URL: https://issues.apache.org/jira/browse/KYLIN-3601
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v2.5.0
> Reporter: huaicui
> Priority: Major
> Attachments: FirstResponseDistribute.jpg,
> SixthResponseDistribute.jpg, image-2018-09-28-15-14-00-288.png, image.png
>
>
> 因为并发性能不够，使用了magang提供的PrepareStatement方法进行测试。性能有所有提高，但随着测试次数的增加，吞吐率会越来越低而且数据超时也越来越多。经过修改代码在queryAndUpdateCache最后返回前加入日志打印：logger.debug("BorrowedCount:"+preparedContextPool.getBorrowedCount()
> +",DestroyedCount:"+preparedContextPool.getDestroyedCount()
> +",CreatedCount:"+preparedContextPool.getCreatedCount()
> +",ReturnedCount:"+preparedContextPool.getReturnedCount()
> 同时配置文件加入该配置：
> kylin.query.statement-cache-max-num-per-key=200
>
>
> 日志显示，当同一sql并发一段时间后，PreparedContextPool创建了越来越多PrepareStatement，并没有进行阻塞后续来的请求。
> !image-2018-09-28-15-14-00-288.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
