Re: select * clause still case all regionserver crash

[hongbin ma]

Hi, are you "qstar" from
https://issues.apache.org/jira/browse/KYLIN-1936?focusedCommentId=15727422&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15727422
? Just want to make sure whether it's a individual case or common case


On Thu, Dec 8, 2016 at 10:00 AM, alaleiwang  wrote:

> about v1.5.4,"select * from table limit N" clause seems not to crash
> regionserver,but meantime no result return for the clause
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6535.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Regards,

*Bin Mahone | 马洪宾*

Re: select * clause still case all regionserver crash

[alaleiwang]

about v1.5.4,"select * from table limit N" clause seems not to crash
regionserver,but meantime no result return for the clause

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6535.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[alaleiwang]

i know it not recommend...

as i said above,some user are not technical;they just click from tableau or
write some test query clause,
while not understanding simple operation will do serious thing,we cannot 
avoid such cases at all

so i still insist on doing something form kylin for  guarantee of such
unrecommened "select *" clause with or without "limit",to make regionserver
running more stable on production enviroment

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6494.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[roger shi]

It’s not recommend to select * without limit. Query server cannot handle too 
many records got from storage.

On 05/12/2016, 10:25 AM, "alaleiwang"  wrote:

thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will 
solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[roger shi]

According to hongbin’s comments in JIRA KYLIN-1936, it fixed limit push down 
issue when dealing with multiple segments.

On 04/12/2016, 7:01 PM, "Alberto Ramón"  wrote:

about KYLIN-1936: what make this change ?
   (keep in mind groups by? keep in mind differents regions ? what is the
new limit ?)


Thanks

2016-12-04 9:39 GMT+01:00 roger shi :

> The limit push down issue is completely resolved in JIRA KYLIN-1936, which
> is applied in 1.5.4. So please try kylin 1.5.4.
>
> On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:
>
> I remember hongbin has further optimization on the limit push down
> after
> 1.5.3; @hongbin, can you confirm this?
>
> 2016-12-02 17:02 GMT+08:00 alaleiwang :
>
> > i can see each regionserver scan over 2000+
> >
> > the query "select *"  is called from tableau used by non-technical
> > analyst,so we can't force them to bypass such clause  or to do
> something
> > from tableau ...
> >
> > how can "limit" work for "select *" without "where"?
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/select-clause-still-cause-all-regionserver-
> > crash-tp6474p6481.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
>

Re: select * clause still case all regionserver crash

[35925138]

sorry, it's my mistake , I click the "load hive table from tree" button , it's 
always empty, 
but when I click "load hive table" button and input the table name ,it's ok, 
and when I click the "load hive table from tree" button again,It's ok too








-- Original --
From:  "alaleiwang";;
Date:  Mon, Dec 5, 2016 10:25 AM
To:  "dev"; 

Subject:  Re: select * clause still case all regionserver crash



thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[alaleiwang]

thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

about KYLIN-1936: what make this change ?
   (keep in mind groups by? keep in mind differents regions ? what is the
new limit ?)


Thanks

2016-12-04 9:39 GMT+01:00 roger shi :

> The limit push down issue is completely resolved in JIRA KYLIN-1936, which
> is applied in 1.5.4. So please try kylin 1.5.4.
>
> On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:
>
> I remember hongbin has further optimization on the limit push down
> after
> 1.5.3; @hongbin, can you confirm this?
>
> 2016-12-02 17:02 GMT+08:00 alaleiwang :
>
> > i can see each regionserver scan over 2000+
> >
> > the query "select *"  is called from tableau used by non-technical
> > analyst,so we can't force them to bypass such clause  or to do
> something
> > from tableau ...
> >
> > how can "limit" work for "select *" without "where"?
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/select-clause-still-cause-all-regionserver-
> > crash-tp6474p6481.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
>

Re: select * clause still case all regionserver crash

[roger shi]

The limit push down issue is completely resolved in JIRA KYLIN-1936, which is 
applied in 1.5.4. So please try kylin 1.5.4.

On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:

I remember hongbin has further optimization on the limit push down after
1.5.3; @hongbin, can you confirm this?

2016-12-02 17:02 GMT+08:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Best regards,

Shaofeng Shi 史少锋

Re: select * clause still case all regionserver crash

[Alberto Ramón]

:)  We have some similiar problem with non-IT
"Why do you want a result table of 1 Millon of rows ¿? "

I don't Know any solution.
The Q is: this is a problem/bug/issue of Kylin ? I think NO, the target of
this is protect to coprocesor (is mandatory)

Select * is not addecuate for Kylin
Impala, Hive, Cassandra, presto, ... Is the best solution for this: View
raw data



2016-12-02 10:02 GMT+01:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: select * clause still case all regionserver crash

[ShaoFeng Shi]

I remember hongbin has further optimization on the limit push down after
1.5.3; @hongbin, can you confirm this?

2016-12-02 17:02 GMT+08:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Best regards,

Shaofeng Shi 史少锋

Re: select * clause still case all regionserver crash

[alaleiwang]

i can see each regionserver scan over 2000+

the query "select *"  is called from tableau used by non-technical
analyst,so we can't force them to bypass such clause  or to do something
from tableau ...

how can "limit" work for "select *" without "where"?




--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6481.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

About select * from   :Kylin 1792
 v1.5.3

Can you check if you see this error: "*Scan row count exceeded threshold:
100, please add filter condition to narrow down backend scan range,
like where clause*"

With Queries like this:

SELECT * FROM TB

There are some limits:

- Read more 1M rows per RS

- Read more 100MB per RD

- Total result of Kylin 3.0 GB (more or less)

- Time limit to solve querie . (I don't remember if it was 10 seconds)


*As resume:* Kylin, can't read Billon of rows, and compute its. Because is
a OLAP Cube and we espect solve queries very very fast (This is the reason
of existence of a Precalculates Cubes)


You must add where condition to Select column from TB Where ...  to make a
predicate pushdown on queries




2016-12-02 3:23 GMT+01:00 alaleiwang :

> KYLIN-1787  say fixed in v1.5.3,but it still happened from v1.5.1 to v1.5.3
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6476.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: select * clause still case all regionserver crash

[alaleiwang]

KYLIN-1787  say fixed in v1.5.3,but it still happened from v1.5.1 to v1.5.3

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6476.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

Sounds like:
KYLIN-1787 

MailList

MailList



2016-12-01 14:04 GMT+01:00 alaleiwang :

> hi:
>i ask the question at
> ：http://apache-kylin.74782.x6.nabble.com/some-question-
> about-setMaxResultSize-for-scanner-CubeHBaseScanRPC-
> CubeHBaseEndpointRPC-td4983.html#a5007
>
>https://issues.apache.org/jira/browse/KYLIN-1787 say it solved the
> problem
>
>but i am still now suffering from the same thing,all regionserver(180+)
> crashed from time to time
> by the clause "select * from table",and limit does not help
>
>the relate cube size is about 3.88TB,and my regionserver memory sumed up
> to 2.04T
>
>this happend from kylin 1.5.1 and kylin 1.5.3,i don't test on kylin
> 1.5.4
>
>more to be added:
>select * from tablename where aa=bb  limit 10 do work,and will not crash
> regionserver
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-case-all-regionserver-crash-tp6474.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>