Hi all, today I also met with the same problem, however, maybe mine is much more strange, the SQL lies in the following: select count(* ) from (select 1 from test1 where condtionx group by col1, col2, col3) t1
since the result of the sub query is greater than 4000000, the exception is thrown out~ however, the final row count of the the whole SQL is just 1 row, such kind of SQL is usually implemented to obtain the total row count of some queries for paging feature~ 2015-05-13 18:15 GMT+08:00 Parkavi Nandagopal <[email protected]>: > After getting that below error (Scan row count exceeded threshold: > 4000000), kylin is stopped/crashed automatically. > Is Kylin single point of Failure? > How to make it has an High availability? > > Thanks, > Parkavi. > > > -----Original Message----- > From: Parkavi Nandagopal > Sent: Wednesday, May 13, 2015 10:49 AM > To: dev; '[email protected]' > Subject: RE: Increase query performance > > Size of my hive fact table = 3.27 GB ( row count 25,236,160) Cube size = > 2.21 GB > > I created hierarchy dimension with 18 levels. > Col1 -> Col2 -> ......upto Col18 > For this 18 levels, total cardinality = 2635 > > I attached 2 log files. > Log1 - query with limit 1000000 > Partial result came. > Log2 - Clicked show all in Query result. > Getting ERROR : exception while executing query: Scan row count exceeded > threshold: 4000000, please add filter condition to narrow down backend scan > range, like where clause. > > Thanks, > Parkavi. > > -----Original Message----- > From: hongbin ma [mailto:[email protected]] > Sent: Wednesday, May 13, 2015 7:15 AM > To: dev > Subject: Re: Increase query performance > > before you expand your cluster, you might need to analyse why it's > delivering poor performance. > > how about the size of your hive fact table? the cardinality of the > dimension columns? > > if possible you can run a query,and paste the query's log in > KYLIN_HOME/logs/kylin.log for that query. we can help you check for any > abnormalities. (make sure you're writing a slightly different query, to > avoid hitting cache) > > On Tue, May 12, 2015 at 2:04 PM, Parkavi Nandagopal <[email protected]> > wrote: > > > Hi , > > > > I have installed kylin and created cube(3GB size) with only one region > > server and when I query the cube data, it is taking much time to show > > the query result in Kylin web UI. > > If I add 3 or more region server node with high configuration and I > > create a cube then query the cube means will it increase the query > performance? > > > > > > Thanks, > > Parkavi. > > > > > > ::DISCLAIMER:: > > > > ---------------------------------------------------------------------- > > ---------------------------------------------------------------------- > > -------- > > > > The contents of this e-mail and any attachment(s) are confidential and > > intended for the named recipient(s) only. > > E-mail transmission is not guaranteed to be secure or error-free as > > information could be intercepted, corrupted, lost, destroyed, arrive > > late or incomplete, or may contain viruses in transmission. The e mail > > and its contents (with or without referred errors) shall therefore not > > attach any liability on the originator or HCL or its affiliates. > > Views or opinions, if any, presented in this email are solely those of > > the author and may not necessarily reflect the views or opinions of > > HCL or its affiliates. Any form of reproduction, dissemination, > > copying, disclosure, modification, distribution and / or publication > > of this message without the prior written consent of authorized > > representative of HCL is strictly prohibited. If you have received > > this email in error please delete it and notify the sender > > immediately. > > Before opening any email and/or attachments, please check them for > > viruses and other defects. > > > > > > ---------------------------------------------------------------------- > > ---------------------------------------------------------------------- > > -------- > > > > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone >
