KANG-SEN LU created KYLIN-3636:
----------------------------------

             Summary: in kylin 2.4.1 and 2.5.1 topn aggregation query caused 
exception
                 Key: KYLIN-3636
                 URL: https://issues.apache.org/jira/browse/KYLIN-3636
             Project: Kylin
          Issue Type: Bug
          Components: Query Engine
    Affects Versions: v2.4.1
            Reporter: KANG-SEN LU


Hi, ShaoFeng:

 

Thanks for the reply. I missed this email and not responded earlier, I am sorry.

 

I tried to reproduce this problem with the sample database, and it did not 
happen.

 

So I am hoping by collecting enough "clue", someone can figure out why this 
problem occurred.

 

--------------------------

I issued the following query at the sample project to exercise the topn 
aggregation:

 

select seller_id, SUM(price) as total from kylin_sales group by seller_id order 
by total limit 5;

 

With my own added debugging, I saw the following log in the kylin.log: (the 
query worked OK).

 

2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] 
model.FunctionDesc:59 : KSL22222, getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_

2018-10-16 16:18:19,963 INFO  [kylin-coproc--pool12-t1] 
v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f: send 
request to the init region server anovadata4.anovadata.local on table 
ANOVA_KYLIN_25X_K758MEAWJG

2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] 
topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName= 
_KY_SUM_KYLIN_SALES_PRICE_

 

 

When I was executing my project query, I issued the following select statement:

 

SELECT  ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\", 
SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT) 
\"vl_aggs_model___USERS_BY_ERROR_3XX\"  FROM  ZETTICSDW.A_VL_HOURLY_V inner 
JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON ( 
ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY = 
ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND 
ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY = 
ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND 
ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY = 
ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY) 

WHERE  ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND 
((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR 
<= '03'))) AND ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'  

GROUP BY  ZETTICSDW.A_VL_HOURLY_V.IMSIID  

ORDER BY  \"vl_aggs_model___USERS_BY_ERROR_3XX\"

LIMIT 25

 

An exception occurred within the method "private ColumnRowType 
buildColumnRowType()" of 
"query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".

 

        if (columns.size() != rowType.getFieldCount()) {

            throw new IllegalStateException("RowType=" + 
rowType.getFieldCount() + ", ColumnRowType=" + columns.size());

        }

 

It printed "RowType=133, ColumnRowType=132".

The RowType list contains one extra column name: "ANY 
_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".

 

 

I think this has something to do with the bug fix "KYLIN-3359 Support 
sum(expression) if possible".

 

After this bug fix was submitted, I noticed that a lot of column name was added 
into rowType like "_KY_SUM_XXX".

 

This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very similar.

 

I also found that this extra column name, only existed in RowType but not in 
ColumnRowType, was added in the method "public void 
implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".

 

With my own debug statement, I saw this debug text in kylin.log:

 

relnode.OLAPJoinRel:362 : KSL54: newField= #132: 
_KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY

 

 

I hope someone with a deeper technical knowledge in kylin query engine can 
figure out what was causing the problem I have seen.

 

Thanks again.

 

Kang-sen

 

*From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]] 
 *Sent:* Friday, October 05, 2018 9:59 PM
 *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
 *Subject:* Re: any body see topn in kylin 2.5.1 working?

 

Hi Kang-Sen,

 

Didn't see this; Can you reproduce the problem with the sample cube? 

 

Kang-Sen Lu <[k...@anovadata.com|mailto:k...@anovadata.com]> 于2018年10月6日周六 
上午12:21写道:

I have been running kylin 2.2.1 with topn aggregation OK.

 

Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am having 
query problem related to topn support.

 

The problem is an exception occurred with the following error message in 
kylin.log:

 

Caused by: java.lang.IllegalStateException: RowType=133, ColumnRowType=132

        at 
org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)

        at 
org.apache.kylin.query.relnode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109)

        at 
org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at 
org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)

        at 
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)

        at 
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)

        at 
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281)

        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)

        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)

        at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801)

        at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656)

        at 
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)

        at 
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)

        at 
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)

        at 
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)

        at 
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)

        ... 84 more

 

 

--

Best regards,

 

Shaofeng Shi 史少锋

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to