I found the setting for the threshold locates in StorageContext.java, the
related piece of codes are:
public class StorageContext {
public static final int HARD_THRESHOLD = 4000000;
thus, I have a question that currently I have already built some segments
successfully, later on, if I change the threshold much greater, will it
affect the existing data in the cube storage?
2015-05-15 18:48 GMT+08:00 dong wang <[email protected]>:
> Hi all, today I also met with the same problem, however, maybe mine is
> much more strange, the SQL lies in the following:
> select count(* ) from (select 1 from test1 where condtionx group by col1,
> col2, col3) t1
>
> since the result of the sub query is greater than 4000000, the exception
> is thrown out~ however, the final row count of the the whole SQL is just 1
> row, such kind of SQL is usually implemented to obtain the total row count
> of some queries for paging feature~
>
> 2015-05-13 18:15 GMT+08:00 Parkavi Nandagopal <[email protected]>:
>
>> After getting that below error (Scan row count exceeded threshold:
>> 4000000), kylin is stopped/crashed automatically.
>> Is Kylin single point of Failure?
>> How to make it has an High availability?
>>
>> Thanks,
>> Parkavi.
>>
>>
>> -----Original Message-----
>> From: Parkavi Nandagopal
>> Sent: Wednesday, May 13, 2015 10:49 AM
>> To: dev; '[email protected]'
>> Subject: RE: Increase query performance
>>
>> Size of my hive fact table = 3.27 GB ( row count 25,236,160) Cube size =
>> 2.21 GB
>>
>> I created hierarchy dimension with 18 levels.
>> Col1 -> Col2 -> ......upto Col18
>> For this 18 levels, total cardinality = 2635
>>
>> I attached 2 log files.
>> Log1 - query with limit 1000000
>> Partial result came.
>> Log2 - Clicked show all in Query result.
>> Getting ERROR : exception while executing query: Scan row count exceeded
>> threshold: 4000000, please add filter condition to narrow down backend scan
>> range, like where clause.
>>
>> Thanks,
>> Parkavi.
>>
>> -----Original Message-----
>> From: hongbin ma [mailto:[email protected]]
>> Sent: Wednesday, May 13, 2015 7:15 AM
>> To: dev
>> Subject: Re: Increase query performance
>>
>> before you expand your cluster, you might need to analyse why it's
>> delivering poor performance.
>>
>> how about the size of your hive fact table? the cardinality of the
>> dimension columns?
>>
>> if possible you can run a query,and paste the query's log in
>> KYLIN_HOME/logs/kylin.log for that query. we can help you check for any
>> abnormalities. (make sure you're writing a slightly different query, to
>> avoid hitting cache)
>>
>> On Tue, May 12, 2015 at 2:04 PM, Parkavi Nandagopal <[email protected]>
>> wrote:
>>
>> > Hi ,
>> >
>> > I have installed kylin and created cube(3GB size) with only one region
>> > server and when I query the cube data, it is taking much time to show
>> > the query result in Kylin web UI.
>> > If I add 3 or more region server node with high configuration and I
>> > create a cube then query the cube means will it increase the query
>> performance?
>> >
>> >
>> > Thanks,
>> > Parkavi.
>> >
>> >
>> > ::DISCLAIMER::
>> >
>> > ----------------------------------------------------------------------
>> > ----------------------------------------------------------------------
>> > --------
>> >
>> > The contents of this e-mail and any attachment(s) are confidential and
>> > intended for the named recipient(s) only.
>> > E-mail transmission is not guaranteed to be secure or error-free as
>> > information could be intercepted, corrupted, lost, destroyed, arrive
>> > late or incomplete, or may contain viruses in transmission. The e mail
>> > and its contents (with or without referred errors) shall therefore not
>> > attach any liability on the originator or HCL or its affiliates.
>> > Views or opinions, if any, presented in this email are solely those of
>> > the author and may not necessarily reflect the views or opinions of
>> > HCL or its affiliates. Any form of reproduction, dissemination,
>> > copying, disclosure, modification, distribution and / or publication
>> > of this message without the prior written consent of authorized
>> > representative of HCL is strictly prohibited. If you have received
>> > this email in error please delete it and notify the sender
>> > immediately.
>> > Before opening any email and/or attachments, please check them for
>> > viruses and other defects.
>> >
>> >
>> > ----------------------------------------------------------------------
>> > ----------------------------------------------------------------------
>> > --------
>> >
>>
>>
>>
>> --
>> Regards,
>>
>> *Bin Mahone | 马洪宾*
>> Apache Kylin: http://kylin.io
>> Github: https://github.com/binmahone
>>
>
>
