[jira] [Updated] (KYLIN-5689) Kylin5 local debugging and simulating prod metadata
[ https://issues.apache.org/jira/browse/KYLIN-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaguang Jia updated KYLIN-5689: --- Attachment: KYLIN-5689 Kylin5本地调试和模拟诊断包中的元数据.pdf > Kylin5 local debugging and simulating prod metadata > --- > > Key: KYLIN-5689 > URL: https://issues.apache.org/jira/browse/KYLIN-5689 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Major > Fix For: 5.0-beta > > Attachments: KYLIN-5689 Kylin5本地调试和模拟诊断包中的元数据.pdf > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5689) Kylin5 local debugging and simulating prod metadata
[ https://issues.apache.org/jira/browse/KYLIN-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaguang Jia updated KYLIN-5689: --- Attachment: (was: KYLIN-5689 Kylin5本地调试和模拟诊断包中的元数据.pdf) > Kylin5 local debugging and simulating prod metadata > --- > > Key: KYLIN-5689 > URL: https://issues.apache.org/jira/browse/KYLIN-5689 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5689) Kylin5 local debugging and simulating prod metadata
[ https://issues.apache.org/jira/browse/KYLIN-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaguang Jia updated KYLIN-5689: --- Attachment: KYLIN-5689 Kylin5本地调试和模拟诊断包中的元数据.pdf > Kylin5 local debugging and simulating prod metadata > --- > > Key: KYLIN-5689 > URL: https://issues.apache.org/jira/browse/KYLIN-5689 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Major > Fix For: 5.0-beta > > Attachments: KYLIN-5689 Kylin5本地调试和模拟诊断包中的元数据.pdf > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5689) Kylin5 local debugging and simulating prod metadata
[ https://issues.apache.org/jira/browse/KYLIN-5689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaguang Jia updated KYLIN-5689: --- Summary: Kylin5 local debugging and simulating prod metadata (was: Kylin5 local debugging and simulating customer metadata) > Kylin5 local debugging and simulating prod metadata > --- > > Key: KYLIN-5689 > URL: https://issues.apache.org/jira/browse/KYLIN-5689 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KYLIN-5689) Kylin5 local debugging and simulating customer metadata
Yaguang Jia created KYLIN-5689: -- Summary: Kylin5 local debugging and simulating customer metadata Key: KYLIN-5689 URL: https://issues.apache.org/jira/browse/KYLIN-5689 Project: Kylin Issue Type: New Feature Components: Metadata Affects Versions: 5.0-alpha Reporter: Yaguang Jia Assignee: Yaguang Jia Fix For: 5.0-beta -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5688) After a system admin user without data permission is added to a user group with data permission for a project, the system admin still has no data permission for this pr
[
https://issues.apache.org/jira/browse/KYLIN-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754492#comment-17754492
]
Yaguang Jia commented on KYLIN-5688:
*Background*
The data permissions of system administrators and regular users are recorded in
different metadata locations.
The data permissions of system administrators are recorded at the global level,
with the metadata path being */_global/sys_acl/user/\{userName}* This records
whether they have global data permissions (configurable in the interface) and
which projects' data permissions they have.
The data permissions of regular users and groups are recorded at the project
level, with the metadata path being */_global/acl/\{projectUuid}* This records
specific data permissions.
*Root Cause*
Currently, when determining if a system administrator (non-super admin) has
specific project's data permission, only two scenarios are considered:
1. Whether the user has global data permission.
2. Whether the user has data permission for that project.
If either condition is met, it is assumed that the system administrator has
data permission for that project. However, it does not consider whether the
user group to which the system administrator belongs has project's data
permission.
*Dev Design*
Add logic to check for system administrator's group-level permissions.
Specifically modify as follows: When it is determined that a system
administrator does not have both global data permission and specific project's
data permission, do not return quickly but continue with subsequent checks for
group-level permissions.
*背景*
系统管理员 与 普通用户 的数据权限记录在不同的元数据位置上
系统管理员的数据权限记录在全局级别，元数据路径为{{{}
/_global/sys_acl/user/\{userName}{}}}，记录了是否拥有全局数据权限（在界面上可以配置），以及拥有哪些项目的数据权限
普通用户和组的数据权限记录在项目级别，元数据路径为 {{{}/_global/acl/\{projectUuid}{}}}，记录了具体的数据权限
*RootCause*
目前判断一个系统管理员(非超级管理员)是否具有特定项目的数据权限时，仅考虑了以下两种情况：
# 该用户是否具有全局数据权限
# 该用户是否具有该项目的数据权限
任意一条成立即认为该系统管理员具有该项目的数据权限
并没有考虑系统管理员所在用户组是否拥有项目的数据权限
*DevDesign*
增加判断系统管理员所在用户组权限的逻辑
具体修改为：当判断到系统管理员不具有 *全局数据权限* 和 *特定项目的数据权限* 时，不做快速返回，继续后续的用户组权限检查
> After a system admin user without data permission is added to a user group
> with data permission for a project, the system admin still has no data
> permission for this project
> -
>
> Key: KYLIN-5688
> URL: https://issues.apache.org/jira/browse/KYLIN-5688
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: 5.0-alpha
>Reporter: Yaguang Jia
>Assignee: Zhiting Guo
>Priority: Minor
> Fix For: 5.0-beta
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KYLIN-5688) After a system admin user without data permission is added to a user group with data permission for a project, the system admin still has no data permission for this proj
Yaguang Jia created KYLIN-5688: -- Summary: After a system admin user without data permission is added to a user group with data permission for a project, the system admin still has no data permission for this project Key: KYLIN-5688 URL: https://issues.apache.org/jira/browse/KYLIN-5688 Project: Kylin Issue Type: Bug Components: Security Affects Versions: 5.0-alpha Reporter: Yaguang Jia Assignee: Zhiting Guo Fix For: 5.0-beta -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5687) Remove the log information added abnormally during job execution
[
https://issues.apache.org/jira/browse/KYLIN-5687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754490#comment-17754490
]
Yaguang Jia commented on KYLIN-5687:
h2. RC
The asyn query task class *org.apache.kylin.query.engine.AsyncQueryApplication*
inherits from {*}org.apache.kylin.engine.spark.application.SparkApplication{*}.
These update behaviors are parent class behaviors and are not overridden by
specific subclasses.
While asyn query does not have corresponding task metadata in Kylin5, If the
update-related methods are not rewritten, it will eventually result in a
NullPointerException due to the inability to find task metadata. However, it
does not affect task execution.
The reason for the abnormal log of detecting resources for building tasks is
the same as above.
h2. Fix Design
Rewrite update metadata related methods in subclasses (rewrite to not require
any implementation)
The details are as follows:
1, in the detection of resources step corresponding to the class to rewrite the
wait for resources method
*org.apache.kylin.engine.spark.application.SparkApplication#waiteForResource*
*org.apache.kylin.engine.spark.application.SparkApplication#waiteForResourceSuccess*
2, asyn query tasks in the class to rewrite the wait for resources method, the
task completion of the update method and task failure update method
*org.apache.kylin.engine.spark.application.SparkApplication#waiteForResourceSuccess*
*org.apache.kylin.engine.spark.application.SparkApplication#logJobInfo*
*org.apache.kylin.engine.spark.application.SparkApplication#updateJobErrorInfo*
h3. RC
异步查询任务类org.apache.kylin.query.engine.AsyncQueryApplication继承自org.apache.kylin.engine.spark.application.SparkApplication。issue中的这些更新行为都是父类的行为，具体的子类并没有重写。而异步查询在Kylin5中并没有相对应的任务元数据，如果不重写更新相关的方法，最终会导致找不到任务元数据而报空指针异常。但是，不影响任务执行。构建任务的检测资源异常日志原因同上。
h3. Fix Design
在子类中重写更新元数据相关方法（重写为不需要任何实现）
具体如下：
1、在检测资源step对应的类中重写等待资源方法
org.apache.kylin.engine.spark.application.SparkApplication#waiteForResource
org.apache.kylin.engine.spark.application.SparkApplication#waiteForResourceSuccess
2、异步查询任务的类中重写等待资源方法、任务完成后的更新方法以及任务失败的更新方法
org.apache.kylin.engine.spark.application.SparkApplication#waiteForResourceSuccess
org.apache.kylin.engine.spark.application.SparkApplication#logJobInfo
org.apache.kylin.engine.spark.application.SparkApplication#updateJobErrorInfo
> Remove the log information added abnormally during job execution
>
>
> Key: KYLIN-5687
> URL: https://issues.apache.org/jira/browse/KYLIN-5687
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: 5.0-alpha
>Reporter: Yaguang Jia
>Assignee: sibing.zhang
>Priority: Minor
> Fix For: 5.0-beta
>
>
> Remove the log information added abnormally during job execution
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KYLIN-5687) Remove the log information added abnormally during job execution
Yaguang Jia created KYLIN-5687: -- Summary: Remove the log information added abnormally during job execution Key: KYLIN-5687 URL: https://issues.apache.org/jira/browse/KYLIN-5687 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: 5.0-alpha Reporter: Yaguang Jia Assignee: sibing.zhang Fix For: 5.0-beta Remove the log information added abnormally during job execution -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5474) Kylin5 throw [null Exception] org.h2.Driver in local debug mode
[ https://issues.apache.org/jira/browse/KYLIN-5474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754423#comment-17754423 ] ASF GitHub Bot commented on KYLIN-5474: --- ygjia closed pull request #2101: KYLIN-5474 fix kylin5 local debug mode URL: https://github.com/apache/kylin/pull/2101 > Kylin5 throw [null Exception] org.h2.Driver in local debug mode > - > > Key: KYLIN-5474 > URL: https://issues.apache.org/jira/browse/KYLIN-5474 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Minor > Fix For: 5.0-beta > > Attachments: image (1).png, image (2).png > > > For Kyln5 local debug mode, please refer -> > https://issues.apache.org/jira/browse/KYLIN-5269 > > kylin5 throw [null Exception] org.h2.Driver when run sql in local debug mode > !image (1).png|width=1017,height=417! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kylin] ygjia closed pull request #2101: KYLIN-5474 fix kylin5 local debug mode
ygjia closed pull request #2101: KYLIN-5474 fix kylin5 local debug mode URL: https://github.com/apache/kylin/pull/2101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
