Hi Chun En, did you analysis those “bad” SQLs, to see whether they well matched with the cube design? Kylin doesn't guarantee every query can be returned in a very short time, but 80396 seconds need administrator's attention. If the query is good, hbase is good, memory is enough, CPU is at normal level, you need investigate what's the real bottleneck;
Previous in eBay deployment we encountered an extreme case (37 dimensions, separated into a couple of aggregation groups), when the query cross aggregation groups, the time is very long; Later we identified the bottleneck and made an enhancement in Kylin v1.1; after that we didn't observe such issue. As you already uses Kylin 1.1, I don't think it is this case. You may need do more investigation or provide more detailed information here to analysis. 2015-11-11 17:32 GMT+08:00 nichunen <[email protected]>: > Hi, > > We did a stress test for our kylin server with 100 concurrent queies. It > worked fine at first. But after 1 day, we can't query kylin any more, and > there is log like "query has been running 80396 seconds", many "bad > queries" were hung there. Hbase nodes were still alive, and the cubes and > jobs could still be listed on the pages. To make sure whether it was the > problem of hbase, I restarted hbase, and did a new query, no log from > region server shown hbase received the query, for as we know, a successful > query will create log like "Klin Coprocessor start; Klin Coprocessor > aggregation done". And from the kylin.log, there were still queries hung. > > > Do you know what caused the problem? In our opnion, it may be because: > 1. We use kylin 1.1 on hbase 1.0.1.1(I modified the hbase version in > pom.xml to create the package); > 2. The tomcat max threads setting, we didn't modify any setting in tomcat; > 3. Kylin's problem. > > BTW, we read the code of BadQueryDetector, and it seems a query thread > will be killed only when low available memory and 5 minutes lasted. We > doubt may be this is not very reasonable. > > Best Regards, > > > > George/倪春恩 > > Software Engineer/软件工程师 > > Mobile:+86-13501723787| Fax:+8610-56842040 > > 北京明略软件系统有限公司（www <http://www.semidata.com/>.mininglamp.com） > > 北京市昌平区东小口镇中东路398号中煤建设集团大厦1号楼4层 > > F4,1#,Zhongmei Construction Group Plaza,398# Zhongdong Road,Changping > District,Beijing,102218 > > > ---------------------------------------------------------------------------------------------------------------------------- > > [image: cid:[email protected]] > -- Best regards, Shaofeng Shi
