The first change is in janino-2.7.6.jar(group org.codehaus.janino). The second change is in calcite-core-1.11.0.jar. After performance test we have not found any concurrency issue. In fact, the change had not solve the problem finally, we are looking for another better solution. The follow is three lines of a query log. The first line and the second line has 32s waits. 2017-06-07 16:02:04,968 INFO [Query cb5c8e7d-c1c0-411a-a463-347c23ea0486-7624] routing.QueryRouter:68 : The realizations remaining: [SalesProjectCube(CUBE)] And the final chosen one is the first one 2017-06-07 16:02:36,533 DEBUG [Query cb5c8e7d-c1c0-411a-a463-347c23ea0486-7624] enumerator.OLAPEnumerator:109 : query storage... 2017-06-07 16:02:36,551 INFO [Query cb5c8e7d-c1c0-411a-a463-347c23ea0486-7624] gtrecord.GTCubeStorageQueryBase:109 : Cuboid identified: cube=SalesProjectCube, cuboidId=1, groupsD=[], otherDimsD=[]
-------------------------------------- 中国建设银行厦门开发中心 苏志锋 ________________________________ 发件人: ShaoFeng Shi <[email protected]> 发送时间: 2017年6月22日 13:20:35 收件人: user; dev 主题: Re: Apache Kylin performance problem Hi Zhifeng, Thanks for the sharing. A few questions for it: Is this change made in Apache Calcite (and which version) ? Will it bring any concurrency issue? If it is safe, that should be merged to Calcite. Thx! 2017-06-22 10:48 GMT+08:00 苏 志锋 <[email protected]<mailto:[email protected]>>: Environment Apache kylin 1.6.0 Jdk1.8.0_72 64-Bit any one cube and any one SQL Problem When the concurrency users count increase, many request’s response time become longer and longer, but some are normal. This is because when calcite parse the sql. The function getBindable(org.apache.calcite.adapter.enumerable.EnumerableInterpretable ,line 119) dynamic create a bindable instance. The bindalbe instance call loadClass(java.lang.ClassLoader line 401) function which has a synchronized java code many times. So, one thead may wait for another. After I change the following code, the performance become much better. 1,org.codehaus.janino.ByteArrayClassLoader +static { + ClassLoader.registerAsParallelCapable(); + } 2,org.apache.calcite.adapter.enumerable.EnumerableRelImplementor line 159 return Expressions.classDecl(Modifier.PUBLIC, -line 160 "Baz", +line 160 "Baz"+System.currentTimeMillis(), line 161 null, line 162 Collections.<Type>singletonList(Bindable.class), line 163 memberDeclarations); -- Best regards, Shaofeng Shi 史少锋
