[jira] [Updated] (KYLIN-2980) Remove getKey/Value setKey/Value from Kylin's Pair.
[ https://issues.apache.org/jira/browse/KYLIN-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao updated KYLIN-2980: -- Description: Pair has no semantic about key/value. And when serializing/deserializing Pair will both has first/second and key/value. > Remove getKey/Value setKey/Value from Kylin's Pair. > --- > > Key: KYLIN-2980 > URL: https://issues.apache.org/jira/browse/KYLIN-2980 > Project: Kylin > Issue Type: Improvement >Reporter: jiatao.tao >Assignee: jiatao.tao > > Pair has no semantic about key/value. And when serializing/deserializing Pair > will both has first/second and key/value. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2980) Remove getKey/Value setKey/Value from Kylin's Pair.
jiatao.tao created KYLIN-2980: - Summary: Remove getKey/Value setKey/Value from Kylin's Pair. Key: KYLIN-2980 URL: https://issues.apache.org/jira/browse/KYLIN-2980 Project: Kylin Issue Type: Improvement Reporter: jiatao.tao Assignee: jiatao.tao -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233611#comment-16233611 ] jiatao.tao commented on KYLIN-2960: --- Hi Shaofeng, jianhua want to sync user's groups in LDAP. We can done this by the code that I comment. > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Major > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226718#comment-16226718 ] Shaofeng SHI commented on KYLIN-2960: - Can someone give a brief subject? > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-2960: - Assignee: peng.jianhua (was: jiatao.tao) > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enab
[
https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226667#comment-16226667
]
jiatao.tao edited comment on KYLIN-2960 at 10/31/17 11:57 AM:
--
Hi peng.jianhua,
1. Kylin need ROLE_ADMIN to indicate that the user is a global ADMIN
{code:java}
@PreAuthorize(Constant.ACCESS_HAS_ROLE_ADMIN)
{code}
(user can define LDAP admin group by kylin.security.acl.admin-role=admin).
2. I'll replace all AuthoritiesPopulator to LDAPAuthoritiesPopulator in
kylinSecurity.xml. But the origin AuthoritiesPopulator will be tagged
@Deprecated but still keeped in case of some previous users use this.
was (Author: aron.tao):
Hi peng.jianhua,
1. Kylin need ROLE_ADMIN to indicate that the user is a global ADMIN(user can
define LDAP admin group by kylin.security.acl.admin-role=admin).
2. I'll replace all AuthoritiesPopulator to LDAPAuthoritiesPopulator in
kylinSecurity.xml. But the origin AuthoritiesPopulator will be tagged
@Deprecated but still keeped in case of some previous users use this.
> We should submit a new feature that it support the authentication for user
> and role and the authentication for user and group when the LDAP
> authentication was enabled.
> ---
>
> Key: KYLIN-2960
> URL: https://issues.apache.org/jira/browse/KYLIN-2960
> Project: Kylin
> Issue Type: New Feature
> Components: General
>Reporter: peng.jianhua
>Assignee: jiatao.tao
> Labels: patch
> Attachments:
> 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch
>
>
> Currently, the user authentication interface that was provided by kylin to
> the third party only supports user and role authentication. However only user
> and group have authentication function when we use the LDAP authentication.
> In fact the authentication for user and role and the authentication for user
> and group have the same functional characteristics between different
> appplication system. So we should submit a new feature that it support the
> authentication for user and role and the authentication for user and group
> when the LDAP authentication was enabled.
> We supplied the checkPermission interface to implement the new feature. In
> the interface we set user groups information to the userRoles parameter when
> the LDAP was enabled, on the contrary we set user roles information to the
> userRoles parameter. The interface is as following:
> /**
> * Checks if a user has permission on an entity.
> *
> * @param user
> * @param userRoles
> * @param entityType String constants defined in AclEntityType
> * @param entityUuid
> * @param permission
> *
> * @return true if has permission
> */
> abstract public boolean checkPermission(String user, List userRoles,
> //
> String entityType, String entityUuid, Permission permission);
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enab
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226667#comment-16226667 ] jiatao.tao edited comment on KYLIN-2960 at 10/31/17 11:56 AM: -- Hi peng.jianhua, 1. Kylin need ROLE_ADMIN to indicate that the user is a global ADMIN(user can define LDAP admin group by kylin.security.acl.admin-role=admin). 2. I'll replace all AuthoritiesPopulator to LDAPAuthoritiesPopulator in kylinSecurity.xml. But the origin AuthoritiesPopulator will be tagged @Deprecated but still keeped in case of some previous users use this. was (Author: aron.tao): Hi peng.jianhua, 1. Kylin need ROLE_ADMIN to indicate that the user is a global ADMIN(see kylin.security.acl.admin-role=admin). 2. I'll replace all AuthoritiesPopulator to LDAPAuthoritiesPopulator in kylinSecurity.xml. But the origin AuthoritiesPopulator will still keep in case of some previous users use this > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: jiatao.tao > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-2960: - Assignee: jiatao.tao (was: peng.jianhua) > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: jiatao.tao > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226667#comment-16226667 ] jiatao.tao commented on KYLIN-2960: --- Hi peng.jianhua, 1. Kylin need ROLE_ADMIN to indicate that the user is a global ADMIN(see kylin.security.acl.admin-role=admin). 2. I'll replace all AuthoritiesPopulator to LDAPAuthoritiesPopulator in kylinSecurity.xml. But the origin AuthoritiesPopulator will still keep in case of some previous users use this > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-2960: - Assignee: peng.jianhua (was: jiatao.tao) > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2960) We should submit a new feature that it support the authentication for user and role and the authentication for user and group when the LDAP authentication was enabled.
[ https://issues.apache.org/jira/browse/KYLIN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiatao.tao reassigned KYLIN-2960: - Assignee: jiatao.tao (was: peng.jianhua) > We should submit a new feature that it support the authentication for user > and role and the authentication for user and group when the LDAP > authentication was enabled. > --- > > Key: KYLIN-2960 > URL: https://issues.apache.org/jira/browse/KYLIN-2960 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: peng.jianhua >Assignee: jiatao.tao > Labels: patch > Attachments: > 0001-KYLIN-2960-We-should-submit-a-new-feature-that-it-su.patch > > > Currently, the user authentication interface that was provided by kylin to > the third party only supports user and role authentication. However only user > and group have authentication function when we use the LDAP authentication. > In fact the authentication for user and role and the authentication for user > and group have the same functional characteristics between different > appplication system. So we should submit a new feature that it support the > authentication for user and role and the authentication for user and group > when the LDAP authentication was enabled. > We supplied the checkPermission interface to implement the new feature. In > the interface we set user groups information to the userRoles parameter when > the LDAP was enabled, on the contrary we set user roles information to the > userRoles parameter. The interface is as following: > /** > * Checks if a user has permission on an entity. > * > * @param user > * @param userRoles > * @param entityType String constants defined in AclEntityType > * @param entityUuid > * @param permission > * > * @return true if has permission > */ > abstract public boolean checkPermission(String user, List userRoles, > // > String entityType, String entityUuid, Permission permission); -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2773) Should not push down join condition related columns are compatible while not consistent
[
https://issues.apache.org/jira/browse/KYLIN-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226471#comment-16226471
]
Roger Shi commented on KYLIN-2773:
--
A little modification is applied on the patch, but the main logic is the same.
> Should not push down join condition related columns are compatible while not
> consistent
> ---
>
> Key: KYLIN-2773
> URL: https://issues.apache.org/jira/browse/KYLIN-2773
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.2.0
>
> Attachments: APACHE-KYLIN-2773.patch
>
>
> For sql,
> {code}
> select PART_DT, META_CATEG_NAME, sum(price)
> from KYLIN_SALES
> INNER JOIN KYLIN_CATEGORY_GROUPINGS ON KYLIN_SALES.LEAF_CATEG_ID =
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID
> AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
> INNER JOIN KYLIN_CAL_DT ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT
> group by PART_DT, META_CATEG_NAME
> order by PART_DT, META_CATEG_NAME
> {code}
> the datatype of KYLIN_SALES.LEAF_CATEG_ID is bigint, while the one of
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID is integer.
> Then the plan transformed by kylin is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(PART_DT=[$0], META_CATEG_NAME=[$16], PRICE=[$5])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPProjectRel(expr#0..19=[{inputs}], PART_DT=[$t0],
> LSTG_FORMAT_NAME=[$t1], SLR_SEGMENT_CD=[$t2], LEAF_CATEG_ID=[$t3],
> LSTG_SITE_ID=[$t4], PRICE=[$t5], SELLER_ID=[$t6], COUNT__=[$t7],
> MIN_PRICE_=[$t8], COUNT_DISTINCT_SELLER_ID_=[$t9],
> USER_DEFINED_FIELD1=[$t10], USER_DEFINED_FIELD3=[$t11], UPD_DATE=[$t12],
> UPD_USER=[$t13], LEAF_CATEG_ID0=[$t14], SITE_ID=[$t15],
> META_CATEG_NAME=[$t16], CATEG_LVL2_NAME=[$t17], CATEG_LVL3_NAME=[$t18])
> OLAPJoinRel(condition=[AND(=($3, $19), =($4, $15))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1,
> 2, 3, 4, 5, 6, 7, 8, 9]])
> OLAPProjectRel(expr#0..8=[{inputs}],
> expr#9=[CAST($t4):BIGINT], USER_DEFINED_FIELD1=[$t0],
> USER_DEFINED_FIELD3=[$t1], UPD_DATE=[$t2], UPD_USER=[$t3],
> LEAF_CATEG_ID=[$t4], SITE_ID=[$t5], META_CATEG_NAME=[$t6],
> CATEG_LVL2_NAME=[$t7], CATEG_LVL3_NAME=[$t8], LEAF_CATEG_ID9=[$t9])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3]])
> {code}
> However, what we expect is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(expr#0..21=[{inputs}], PART_DT=[$t0],
> META_CATEG_NAME=[$t17], PRICE=[$t4])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPJoinRel(condition=[AND(=($2, $10), =($3, $11))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9]])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3, 4]])
> {code}
> The reason for this difference is as follows:
> * Although we remove the {{JoinPushExpressionsRule}} in {{OLAPTableScan}},
> the method {{RelOptUtil.pushDownJoinConditions()}} is still invoked when
> creating a join in {{SqlToRelConverter}}.
> * In the method of {{RelOptUtil.pushDownJoinConditions()}}, since the
> datatypes of the join related columns are not the same, *cast* function will
> be automatically assigned to KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID. Then a
> {{OLAPProjectRel}} will be introduced.
> In kylin, we don't need {{RelOptUtil.pushDownJoinConditions()}}. Therefore,
> the solution for this is just remove that logic.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2773) Should not push down join condition related columns are compatible while not consistent
[
https://issues.apache.org/jira/browse/KYLIN-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2773:
Fix Version/s: (was: v2.3.0)
v2.2.0
Roger told me the patch has been merged with commit:
87f1732b0a26191122788baaa2fc9d97c96210da
> Should not push down join condition related columns are compatible while not
> consistent
> ---
>
> Key: KYLIN-2773
> URL: https://issues.apache.org/jira/browse/KYLIN-2773
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.2.0
>
> Attachments: APACHE-KYLIN-2773.patch
>
>
> For sql,
> {code}
> select PART_DT, META_CATEG_NAME, sum(price)
> from KYLIN_SALES
> INNER JOIN KYLIN_CATEGORY_GROUPINGS ON KYLIN_SALES.LEAF_CATEG_ID =
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID
> AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
> INNER JOIN KYLIN_CAL_DT ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT
> group by PART_DT, META_CATEG_NAME
> order by PART_DT, META_CATEG_NAME
> {code}
> the datatype of KYLIN_SALES.LEAF_CATEG_ID is bigint, while the one of
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID is integer.
> Then the plan transformed by kylin is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(PART_DT=[$0], META_CATEG_NAME=[$16], PRICE=[$5])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPProjectRel(expr#0..19=[{inputs}], PART_DT=[$t0],
> LSTG_FORMAT_NAME=[$t1], SLR_SEGMENT_CD=[$t2], LEAF_CATEG_ID=[$t3],
> LSTG_SITE_ID=[$t4], PRICE=[$t5], SELLER_ID=[$t6], COUNT__=[$t7],
> MIN_PRICE_=[$t8], COUNT_DISTINCT_SELLER_ID_=[$t9],
> USER_DEFINED_FIELD1=[$t10], USER_DEFINED_FIELD3=[$t11], UPD_DATE=[$t12],
> UPD_USER=[$t13], LEAF_CATEG_ID0=[$t14], SITE_ID=[$t15],
> META_CATEG_NAME=[$t16], CATEG_LVL2_NAME=[$t17], CATEG_LVL3_NAME=[$t18])
> OLAPJoinRel(condition=[AND(=($3, $19), =($4, $15))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1,
> 2, 3, 4, 5, 6, 7, 8, 9]])
> OLAPProjectRel(expr#0..8=[{inputs}],
> expr#9=[CAST($t4):BIGINT], USER_DEFINED_FIELD1=[$t0],
> USER_DEFINED_FIELD3=[$t1], UPD_DATE=[$t2], UPD_USER=[$t3],
> LEAF_CATEG_ID=[$t4], SITE_ID=[$t5], META_CATEG_NAME=[$t6],
> CATEG_LVL2_NAME=[$t7], CATEG_LVL3_NAME=[$t8], LEAF_CATEG_ID9=[$t9])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3]])
> {code}
> However, what we expect is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(expr#0..21=[{inputs}], PART_DT=[$t0],
> META_CATEG_NAME=[$t17], PRICE=[$t4])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPJoinRel(condition=[AND(=($2, $10), =($3, $11))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9]])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3, 4]])
> {code}
> The reason for this difference is as follows:
> * Although we remove the {{JoinPushExpressionsRule}} in {{OLAPTableScan}},
> the method {{RelOptUtil.pushDownJoinConditions()}} is still invoked when
> creating a join in {{SqlToRelConverter}}.
> * In the method of {{RelOptUtil.pushDownJoinConditions()}}, since the
> datatypes of the join related columns are not the same, *cast* function will
> be automatically assigned to KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID. Then a
> {{OLAPProjectRel}} will be introduced.
> In kylin, we don't need {{RelOptUtil.pushDownJoinConditions()}}. Therefore,
> the solution for this is just remove that logic.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2773) Should not push down join condition related columns are compatible while not consistent
[
https://issues.apache.org/jira/browse/KYLIN-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2773.
-
Resolution: Fixed
> Should not push down join condition related columns are compatible while not
> consistent
> ---
>
> Key: KYLIN-2773
> URL: https://issues.apache.org/jira/browse/KYLIN-2773
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.2.0
>
> Attachments: APACHE-KYLIN-2773.patch
>
>
> For sql,
> {code}
> select PART_DT, META_CATEG_NAME, sum(price)
> from KYLIN_SALES
> INNER JOIN KYLIN_CATEGORY_GROUPINGS ON KYLIN_SALES.LEAF_CATEG_ID =
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID
> AND KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
> INNER JOIN KYLIN_CAL_DT ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT
> group by PART_DT, META_CATEG_NAME
> order by PART_DT, META_CATEG_NAME
> {code}
> the datatype of KYLIN_SALES.LEAF_CATEG_ID is bigint, while the one of
> KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID is integer.
> Then the plan transformed by kylin is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(PART_DT=[$0], META_CATEG_NAME=[$16], PRICE=[$5])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPProjectRel(expr#0..19=[{inputs}], PART_DT=[$t0],
> LSTG_FORMAT_NAME=[$t1], SLR_SEGMENT_CD=[$t2], LEAF_CATEG_ID=[$t3],
> LSTG_SITE_ID=[$t4], PRICE=[$t5], SELLER_ID=[$t6], COUNT__=[$t7],
> MIN_PRICE_=[$t8], COUNT_DISTINCT_SELLER_ID_=[$t9],
> USER_DEFINED_FIELD1=[$t10], USER_DEFINED_FIELD3=[$t11], UPD_DATE=[$t12],
> UPD_USER=[$t13], LEAF_CATEG_ID0=[$t14], SITE_ID=[$t15],
> META_CATEG_NAME=[$t16], CATEG_LVL2_NAME=[$t17], CATEG_LVL3_NAME=[$t18])
> OLAPJoinRel(condition=[AND(=($3, $19), =($4, $15))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1,
> 2, 3, 4, 5, 6, 7, 8, 9]])
> OLAPProjectRel(expr#0..8=[{inputs}],
> expr#9=[CAST($t4):BIGINT], USER_DEFINED_FIELD1=[$t0],
> USER_DEFINED_FIELD3=[$t1], UPD_DATE=[$t2], UPD_USER=[$t3],
> LEAF_CATEG_ID=[$t4], SITE_ID=[$t5], META_CATEG_NAME=[$t6],
> CATEG_LVL2_NAME=[$t7], CATEG_LVL3_NAME=[$t8], LEAF_CATEG_ID9=[$t9])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3]])
> {code}
> However, what we expect is as follows:
> {code}
> OLAPToEnumerableConverter
> OLAPLimitRel(fetch=[5])
> OLAPSortRel(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC])
> OLAPAggregateRel(group=[{0, 1}], EXPR$2=[SUM($2)])
> OLAPProjectRel(expr#0..21=[{inputs}], PART_DT=[$t0],
> META_CATEG_NAME=[$t17], PRICE=[$t4])
> OLAPJoinRel(condition=[=($0, $19)], joinType=[inner])
> {code}
> {color:#f79232}
> {code}
> OLAPJoinRel(condition=[AND(=($2, $10), =($3, $11))],
> joinType=[inner])
> OLAPTableScan(table=[[DEFAULT, KYLIN_SALES]], fields=[[0, 1, 2,
> 3, 4, 5, 6, 7, 8, 9]])
> {code}
> {color}
> {code}
> OLAPTableScan(table=[[DEFAULT, KYLIN_CATEGORY_GROUPINGS]],
> fields=[[0, 1, 2, 3, 4, 5, 6, 7, 8]])
> OLAPTableScan(table=[[DEFAULT, KYLIN_CAL_DT]], fields=[[0, 1, 2,
> 3, 4]])
> {code}
> The reason for this difference is as follows:
> * Although we remove the {{JoinPushExpressionsRule}} in {{OLAPTableScan}},
> the method {{RelOptUtil.pushDownJoinConditions()}} is still invoked when
> creating a join in {{SqlToRelConverter}}.
> * In the method of {{RelOptUtil.pushDownJoinConditions()}}, since the
> datatypes of the join related columns are not the same, *cast* function will
> be automatically assigned to KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID. Then a
> {{OLAPProjectRel}} will be introduced.
> In kylin, we don't need {{RelOptUtil.pushDownJoinConditions()}}. Therefore,
> the solution for this is just remove that logic.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
