[
https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688068#comment-16688068
]
KANG-SEN LU commented on KYLIN-3636:
------------------------------------
Hi, Shaofeng:
I would like to add that there is NO bug in the kylin code, one may argue.
Because the reason I faced those problem related to TOPN aggregation function
is caused by "improper documentation about how to configure TOPN metric". So if
we can improve the usage documentation, there will be more happy users.
> in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
> ----------------------------------------------------------------
>
> Key: KYLIN-3636
> URL: https://issues.apache.org/jira/browse/KYLIN-3636
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v2.4.1
> Reporter: KANG-SEN LU
> Assignee: Shaofeng SHI
> Priority: Major
> Fix For: v2.6.0
>
>
> Hi, ShaoFeng:
>
> Thanks for the reply. I missed this email and not responded earlier, I am
> sorry.
>
> I tried to reproduce this problem with the sample database, and it did not
> happen.
>
> So I am hoping by collecting enough "clue", someone can figure out why this
> problem occurred.
>
> --------------------------
> I issued the following query at the sample project to exercise the topn
> aggregation:
>
> select seller_id, SUM(price) as total from kylin_sales group by seller_id
> order by total limit 5;
>
> With my own added debugging, I saw the following log in the kylin.log: (the
> query worked OK).
>
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> model.FunctionDesc:59 : KSL22222,
> getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO [kylin-coproc--pool12-t1]
> v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
> send request to the init region server anovadata4.anovadata.local on table
> ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90]
> topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFieldName=
> _KY_SUM_KYLIN_SALES_PRICE_
>
>
> When I was executing my project query, I issued the following select
> statement:
>
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\",
> SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
> \"vl_aggs_model___USERS_BY_ERROR_3XX\" FROM ZETTICSDW.A_VL_HOURLY_V inner
> JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V ON (
> ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND
> ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY)
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND
> ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '02') AND
> (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND
> ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3%'
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID
> ORDER BY \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>
> An exception occurred within the method "private ColumnRowType
> buildColumnRowType()" of
> "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>
> if (columns.size() != rowType.getFieldCount()) {
> throw new IllegalStateException("RowType=" +
> rowType.getFieldCount() + ", ColumnRowType=" + columns.size());
> }
>
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY
> _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>
>
> I think this has something to do with the bug fix "KYLIN-3359 Support
> sum(expression) if possible".
>
> After this bug fix was submitted, I noticed that a lot of column name was
> added into rowType like "_KY_SUM_XXX".
>
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very
> similar.
>
> I also found that this extra column name, only existed in RowType but not in
> ColumnRowType, was added in the method "public void
> implementRewrite(RewriteImplementor implementor)" within "OLAPJoinRel.java".
>
> With my own debug statement, I saw this debug text in kylin.log:
>
> relnode.OLAPJoinRel:362 : KSL54: newField= #132:
> _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY
>
>
> I hope someone with a deeper technical knowledge in kylin query engine can
> figure out what was causing the problem I have seen.
>
> Thanks again.
>
> Kang-sen
>
> *From:* ShaoFeng Shi [[mailto:shaofeng...@apache.org]]
> *Sent:* Friday, October 05, 2018 9:59 PM
> *To:* user <[u...@kylin.apache.org|mailto:u...@kylin.apache.org]>
> *Subject:* Re: any body see topn in kylin 2.5.1 working?
>
> Hi Kang-Sen,
>
> Didn't see this; Can you reproduce the problem with the sample cube?
>
> Kang-Sen Lu <[k...@anovadata.com|mailto:k...@anovadata.com]> 于2018年10月6日周六
> 上午12:21写道：
> I have been running kylin 2.2.1 with topn aggregation OK.
>
> Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am having
> query problem related to topn support.
>
> The problem is an exception occurred with the following error message in
> kylin.log:
>
> Caused by: java.lang.IllegalStateException: RowType=133, ColumnRowType=132
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109)
> at
> org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)
> at
> org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 84 more
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
