[jira] [Commented] (KYLIN-5823) improve text recognition dialog of add batch dimensions
[ https://issues.apache.org/jira/browse/KYLIN-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835995#comment-17835995 ] ASF GitHub Bot commented on KYLIN-5823: --- liyang-kylin merged PR #2180: URL: https://github.com/apache/kylin/pull/2180 > improve text recognition dialog of add batch dimensions > --- > > Key: KYLIN-5823 > URL: https://issues.apache.org/jira/browse/KYLIN-5823 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > Attachments: image-2024-04-11-11-42-40-903.png, > image-2024-04-11-11-43-11-768.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5825) fix aggregate modal and table index modal ui and fix chrome 50 layout bug
[ https://issues.apache.org/jira/browse/KYLIN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835993#comment-17835993 ] ASF GitHub Bot commented on KYLIN-5825: --- liyang-kylin merged PR #2176: URL: https://github.com/apache/kylin/pull/2176 > fix aggregate modal and table index modal ui and fix chrome 50 layout bug > - > > Key: KYLIN-5825 > URL: https://issues.apache.org/jira/browse/KYLIN-5825 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > Attachments: image-2024-04-11-12-00-01-336.png, > image-2024-04-11-12-00-56-539.png, image-2024-04-11-12-02-44-264.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5823) improve text recognition dialog of add batch dimensions
[ https://issues.apache.org/jira/browse/KYLIN-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835980#comment-17835980 ] ASF GitHub Bot commented on KYLIN-5823: --- anyisky opened a new pull request, #2180: URL: https://github.com/apache/kylin/pull/2180 (no comment) > improve text recognition dialog of add batch dimensions > --- > > Key: KYLIN-5823 > URL: https://issues.apache.org/jira/browse/KYLIN-5823 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > Attachments: image-2024-04-11-11-42-40-903.png, > image-2024-04-11-11-43-11-768.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5822) support history table adds left-closed and right-closed time interval
[ https://issues.apache.org/jira/browse/KYLIN-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835929#comment-17835929 ] ASF GitHub Bot commented on KYLIN-5822: --- liyang-kylin merged PR #2178: URL: https://github.com/apache/kylin/pull/2178 > support history table adds left-closed and right-closed time interval > - > > Key: KYLIN-5822 > URL: https://issues.apache.org/jira/browse/KYLIN-5822 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5824) refine handel error for export models
[ https://issues.apache.org/jira/browse/KYLIN-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835925#comment-17835925 ] ASF GitHub Bot commented on KYLIN-5824: --- liyang-kylin merged PR #2175: URL: https://github.com/apache/kylin/pull/2175 > refine handel error for export models > - > > Key: KYLIN-5824 > URL: https://issues.apache.org/jira/browse/KYLIN-5824 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > The system reports error when the number of models exceeds 256 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5823) improve text recognition dialog of add batch dimensions
[ https://issues.apache.org/jira/browse/KYLIN-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835923#comment-17835923 ] ASF GitHub Bot commented on KYLIN-5823: --- liyang-kylin merged PR #2174: URL: https://github.com/apache/kylin/pull/2174 > improve text recognition dialog of add batch dimensions > --- > > Key: KYLIN-5823 > URL: https://issues.apache.org/jira/browse/KYLIN-5823 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5826) fix table index select all issue and fix load user or usergroup list filter issue
[ https://issues.apache.org/jira/browse/KYLIN-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835927#comment-17835927 ] ASF GitHub Bot commented on KYLIN-5826: --- liyang-kylin merged PR #2177: URL: https://github.com/apache/kylin/pull/2177 > fix table index select all issue and fix load user or usergroup list filter > issue > - > > Key: KYLIN-5826 > URL: https://issues.apache.org/jira/browse/KYLIN-5826 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Critical > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5821) fix model edit page pfk flag bug and enhance model canvas window position
[ https://issues.apache.org/jira/browse/KYLIN-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835921#comment-17835921 ] ASF GitHub Bot commented on KYLIN-5821: --- liyang-kylin merged PR #2172: URL: https://github.com/apache/kylin/pull/2172 > fix model edit page pfk flag bug and enhance model canvas window position > - > > Key: KYLIN-5821 > URL: https://issues.apache.org/jira/browse/KYLIN-5821 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5819) optimize the warning message on the dialog for editing computed column
[ https://issues.apache.org/jira/browse/KYLIN-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835919#comment-17835919 ] ASF GitHub Bot commented on KYLIN-5819: --- liyang-kylin merged PR #2170: URL: https://github.com/apache/kylin/pull/2170 > optimize the warning message on the dialog for editing computed column > -- > > Key: KYLIN-5819 > URL: https://issues.apache.org/jira/browse/KYLIN-5819 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5813) improve query history page filter item and model list page button text
[ https://issues.apache.org/jira/browse/KYLIN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835095#comment-17835095 ] ASF GitHub Bot commented on KYLIN-5813: --- liyang-kylin merged PR #2169: URL: https://github.com/apache/kylin/pull/2169 > improve query history page filter item and model list page button text > -- > > Key: KYLIN-5813 > URL: https://issues.apache.org/jira/browse/KYLIN-5813 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Major > Fix For: 5.0.0 > > Attachments: image-2024-04-09-09-38-42-450.png, > image-2024-04-09-09-39-13-571.png, image-2024-04-09-09-39-40-593.png, > image-2024-04-09-09-44-16-515.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5820) added pagination and add count distinct function in batch measure dialog
[ https://issues.apache.org/jira/browse/KYLIN-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835097#comment-17835097 ] ASF GitHub Bot commented on KYLIN-5820: --- liyang-kylin merged PR #2171: URL: https://github.com/apache/kylin/pull/2171 > added pagination and add count distinct function in batch measure dialog > - > > Key: KYLIN-5820 > URL: https://issues.apache.org/jira/browse/KYLIN-5820 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Critical > Fix For: 5.0.0 > > Attachments: image-2024-04-09-09-54-05-858.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5822) scd issue
[ https://issues.apache.org/jira/browse/KYLIN-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834508#comment-17834508 ] ASF GitHub Bot commented on KYLIN-5822: --- anyisky opened a new pull request, #2178: URL: https://github.com/apache/kylin/pull/2178 (no comment) > scd issue > - > > Key: KYLIN-5822 > URL: https://issues.apache.org/jira/browse/KYLIN-5822 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > Unable to save after modifying the Shardby column of the existing table index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5822) only edit shardby disable save issue
[ https://issues.apache.org/jira/browse/KYLIN-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834504#comment-17834504 ] ASF GitHub Bot commented on KYLIN-5822: --- anyisky closed pull request #2173: KYLIN-5822,only edit shardby disable save issue(cp from 4.6.16) URL: https://github.com/apache/kylin/pull/2173 > only edit shardby disable save issue > > > Key: KYLIN-5822 > URL: https://issues.apache.org/jira/browse/KYLIN-5822 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > Unable to save after modifying the Shardby column of the existing table index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5826) fix table index select all issue and fix load user or usergroup list filter issue
[ https://issues.apache.org/jira/browse/KYLIN-5826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834497#comment-17834497 ] ASF GitHub Bot commented on KYLIN-5826: --- anyisky opened a new pull request, #2177: URL: https://github.com/apache/kylin/pull/2177 …group list filter issue > fix table index select all issue and fix load user or usergroup list filter > issue > - > > Key: KYLIN-5826 > URL: https://issues.apache.org/jira/browse/KYLIN-5826 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Critical > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5825) clear all mandatory
[ https://issues.apache.org/jira/browse/KYLIN-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834228#comment-17834228 ] ASF GitHub Bot commented on KYLIN-5825: --- anyisky opened a new pull request, #2176: URL: https://github.com/apache/kylin/pull/2176 (no comment) > clear all mandatory > --- > > Key: KYLIN-5825 > URL: https://issues.apache.org/jira/browse/KYLIN-5825 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > Support clearing required dimensions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5824) refine handel error for export models
[ https://issues.apache.org/jira/browse/KYLIN-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834221#comment-17834221 ] ASF GitHub Bot commented on KYLIN-5824: --- anyisky opened a new pull request, #2175: URL: https://github.com/apache/kylin/pull/2175 (no comment) > refine handel error for export models > - > > Key: KYLIN-5824 > URL: https://issues.apache.org/jira/browse/KYLIN-5824 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > The system reports error when the number of models exceeds 256 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5823) recognize add dimensions for model edit and text Recognize table index issue
[ https://issues.apache.org/jira/browse/KYLIN-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834213#comment-17834213 ] ASF GitHub Bot commented on KYLIN-5823: --- anyisky opened a new pull request, #2174: URL: https://github.com/apache/kylin/pull/2174 …e table index issue > recognize add dimensions for model edit and text Recognize table index issue > - > > Key: KYLIN-5823 > URL: https://issues.apache.org/jira/browse/KYLIN-5823 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > feature: > Support adding dimensions in batches using the text recognition function when > setting dimensions on the model editing page > bug: > When adding a table index, the text recognition function cannot be used to > identify the columns contained in the measure -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5822) only edit shardby disable save issue
[ https://issues.apache.org/jira/browse/KYLIN-5822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834201#comment-17834201 ] ASF GitHub Bot commented on KYLIN-5822: --- anyisky opened a new pull request, #2173: URL: https://github.com/apache/kylin/pull/2173 (no comment) > only edit shardby disable save issue > > > Key: KYLIN-5822 > URL: https://issues.apache.org/jira/browse/KYLIN-5822 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > Fix For: 5.0.0 > > > Unable to save after modifying the Shardby column of the existing table index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5821) refine setTableInView issue
[ https://issues.apache.org/jira/browse/KYLIN-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834191#comment-17834191 ] ASF GitHub Bot commented on KYLIN-5821: --- anyisky opened a new pull request, #2172: URL: https://github.com/apache/kylin/pull/2172 (no comment) > refine setTableInView issue > --- > > Key: KYLIN-5821 > URL: https://issues.apache.org/jira/browse/KYLIN-5821 > Project: Kylin > Issue Type: Bug >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > > On the model editing page, the positioning of the "Show Table" function in > the search bar is not accurate -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5820) batch add measures issue
[ https://issues.apache.org/jira/browse/KYLIN-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834187#comment-17834187 ] ASF GitHub Bot commented on KYLIN-5820: --- anyisky opened a new pull request, #2171: URL: https://github.com/apache/kylin/pull/2171 (no comment) > batch add measures issue > > > Key: KYLIN-5820 > URL: https://issues.apache.org/jira/browse/KYLIN-5820 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Critical > > h1. Batch adding measure pages supports paging display columns to optimize > the page response performance when there are too many columns -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5819) edit cc tips issue
[ https://issues.apache.org/jira/browse/KYLIN-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834172#comment-17834172 ] ASF GitHub Bot commented on KYLIN-5819: --- anyisky opened a new pull request, #2170: URL: https://github.com/apache/kylin/pull/2170 (no comment) > edit cc tips issue > -- > > Key: KYLIN-5819 > URL: https://issues.apache.org/jira/browse/KYLIN-5819 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Minor > > h1. Optimize the copywriting tips that will clear the relevant indexes after > modifying the expression of the computed column -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5813) history filter issue
[ https://issues.apache.org/jira/browse/KYLIN-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834162#comment-17834162 ] ASF GitHub Bot commented on KYLIN-5813: --- anyisky opened a new pull request, #2169: URL: https://github.com/apache/kylin/pull/2169 (no comment) > history filter issue > > > Key: KYLIN-5813 > URL: https://issues.apache.org/jira/browse/KYLIN-5813 > Project: Kylin > Issue Type: Improvement >Reporter: chunyan.huang >Assignee: chunyan.huang >Priority: Major > Fix For: 5.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5748) Skip build flat table for index build job if all new created indexes can be built from existed parent index
[ https://issues.apache.org/jira/browse/KYLIN-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831024#comment-17831024 ] ASF GitHub Bot commented on KYLIN-5748: --- Mrhs121 commented on PR #2162: URL: https://github.com/apache/kylin/pull/2162#issuecomment-2021131685 Your proposal will be invalid in this use case: if you add a new bitmap type metric to the model, you can build the new index separately, and the task type is also INDEX_BUIDL. If you skip building the dictionary, an error will occur when generating encoding columns for the flat table later. > Skip build flat table for index build job if all new created indexes can be > built from existed parent index > --- > > Key: KYLIN-5748 > URL: https://issues.apache.org/jira/browse/KYLIN-5748 > Project: Kylin > Issue Type: Improvement >Affects Versions: 5.0-beta >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Major > > For an index build job, if all the new added indexes can be built from their > existed parent index. No index was built from new created flat table. > For this case, there won't be any data inconsistency across indexes. And > there is no need to build a new flat table to serve the stage of index build > by layer. > So I think we can skip stage BUILD DICT and stage GENERATE FLAT TABLE for an > index build job in condition that all new added indexes can be built from > their existed parent index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-4413) Add canary tool to Kylin
[ https://issues.apache.org/jira/browse/KYLIN-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17826955#comment-17826955 ] ASF GitHub Bot commented on KYLIN-4413: --- zhoukangcn closed pull request #1152: KYLIN-4413: add canary tool URL: https://github.com/apache/kylin/pull/1152 > Add canary tool to Kylin > > > Key: KYLIN-4413 > URL: https://issues.apache.org/jira/browse/KYLIN-4413 > Project: Kylin > Issue Type: Improvement >Reporter: ZhouKang >Priority: Major > > Canary tool can be used to detect service availability, and also can detect > query latency. > Many other hadoop system has canary, such as HBase and HDFS. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-4227) add segment build callback
[ https://issues.apache.org/jira/browse/KYLIN-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17826954#comment-17826954 ] ASF GitHub Bot commented on KYLIN-4227: --- zhoukangcn closed pull request #1008: KYLIN-4227: add segment build callback URL: https://github.com/apache/kylin/pull/1008 > add segment build callback > -- > > Key: KYLIN-4227 > URL: https://issues.apache.org/jira/browse/KYLIN-4227 > Project: Kylin > Issue Type: New Feature >Reporter: ZhouKang >Priority: Major > Attachments: 1.png, 2.png > > > add HTTP callback for cube, some other system could use this feature to add a > hook when segment building done. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5386) 使用Segment元数据完成部分数据过滤加速长时间范围数据查询
[ https://issues.apache.org/jira/browse/KYLIN-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808862#comment-17808862 ] ASF GitHub Bot commented on KYLIN-5386: --- hit-lacus closed pull request #2069: KYLIN-5386 segment过滤完成时间分区过滤从而提升cuboid的查询粒度 URL: https://github.com/apache/kylin/pull/2069 > 使用Segment元数据完成部分数据过滤加速长时间范围数据查询 > --- > > Key: KYLIN-5386 > URL: https://issues.apache.org/jira/browse/KYLIN-5386 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v4.0.3 >Reporter: xjlem >Priority: Major > Attachments: KYLIN-5386.v4.0.3-release.001.patch > > > Query中分区字段影响segement过滤，为此查询中会带上分区字段作限制，但分区字段将会影响cuboid选择，使得cuboid始终是天粒度。为此我们使用segment的元数据信息满足查询的过滤，可以通过segment元数据进行过滤的部分将使用更粗粒度的cuboid进行聚合查询。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5748) Skip build flat table for index build job if all new created indexes can be built from existed parent index
[ https://issues.apache.org/jira/browse/KYLIN-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800428#comment-17800428 ] ASF GitHub Bot commented on KYLIN-5748: --- zznlime commented on PR #2162: URL: https://github.com/apache/kylin/pull/2162#issuecomment-1869324455 @hit-lacus PTAL, Thanks > Skip build flat table for index build job if all new created indexes can be > built from existed parent index > --- > > Key: KYLIN-5748 > URL: https://issues.apache.org/jira/browse/KYLIN-5748 > Project: Kylin > Issue Type: Improvement >Affects Versions: 5.0-beta >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Major > > For an index build job, if all the new added indexes can be built from their > existed parent index. No index was built from new created flat table. > For this case, there won't be any data inconsistency across indexes. And > there is no need to build a new flat table to serve the stage of index build > by layer. > So I think we can skip stage BUILD DICT and stage GENERATE FLAT TABLE for an > index build job in condition that all new added indexes can be built from > their existed parent index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5748) Skip build flat table for index build job if all new created indexes can be built from existed parent index
[ https://issues.apache.org/jira/browse/KYLIN-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800421#comment-17800421 ] ASF GitHub Bot commented on KYLIN-5748: --- zznlime opened a new pull request, #2162: URL: https://github.com/apache/kylin/pull/2162 For an index build job, if all the new added indexes can be built from their existed parent index. No index was built from new created flat table. For this case, there won't be any data inconsistency across indexes. And there is no need to build a new flat table to serve the stage of index build by layer. So I think we can skip stage BUILD DICT and stage GENERATE FLAT TABLE for an index build job in condition that all new added indexes can be built from their existed parent index > Skip build flat table for index build job if all new created indexes can be > built from existed parent index > --- > > Key: KYLIN-5748 > URL: https://issues.apache.org/jira/browse/KYLIN-5748 > Project: Kylin > Issue Type: Improvement >Affects Versions: 5.0-beta >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Major > > For an index build job, if all the new added indexes can be built from their > existed parent index. No index was built from new created flat table. > For this case, there won't be any data inconsistency across indexes. And > there is no need to build a new flat table to serve the stage of index build > by layer. > So I think we can skip stage BUILD DICT and stage GENERATE FLAT TABLE for an > index build job in condition that all new added indexes can be built from > their existed parent index -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798503#comment-17798503
]
ASF GitHub Bot commented on KYLIN-5731:
---
pfzhan commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1431123891
##
src/tool/src/main/java/org/apache/kylin/helper/MetadataToolHelper.java:
##
@@ -374,7 +373,7 @@ private int doRestore(ResourceStore currentResourceStore,
ResourceStore restoreR
}
public void cleanStorage(boolean storageCleanup, List projects,
double requestFSRate,
-int retryTimes) {
+ int retryTimes) {
Review Comment:
这个文件的 format 都有问题
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[ https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798501#comment-17798501 ] ASF GitHub Bot commented on KYLIN-5731: --- pfzhan commented on code in PR #2161: URL: https://github.com/apache/kylin/pull/2161#discussion_r1431121745 ## src/modeling-service/src/main/java/org/apache/kylin/rest/service/MetaStoreService.java: ## @@ -121,19 +91,45 @@ import org.springframework.stereotype.Component; import org.springframework.web.multipart.MultipartFile; -import org.apache.kylin.guava30.shaded.common.collect.Lists; -import org.apache.kylin.guava30.shaded.common.collect.Maps; -import org.apache.kylin.guava30.shaded.common.collect.Sets; -import org.apache.kylin.guava30.shaded.common.io.ByteSource; - -import org.apache.kylin.metadata.recommendation.candidate.JdbcRawRecStore; -import org.apache.kylin.metadata.recommendation.candidate.RawRecItem; -import org.apache.kylin.metadata.recommendation.candidate.RawRecManager; -import org.apache.kylin.metadata.recommendation.ref.OptRecManagerV2; +import javax.servlet.http.HttpServletRequest; +import javax.xml.bind.DatatypeConverter; +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.Charset; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; +import java.util.stream.Stream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; +import java.util.zip.ZipOutputStream; -import lombok.Setter; -import lombok.val; -import lombok.var; +import static org.apache.kylin.common.constant.Constants.KE_VERSION; +import static org.apache.kylin.common.exception.ServerErrorCode.FAILED_CREATE_MODEL; +import static org.apache.kylin.common.exception.ServerErrorCode.MODEL_EXPORT_ERROR; +import static org.apache.kylin.common.exception.ServerErrorCode.MODEL_IMPORT_ERROR; +import static org.apache.kylin.common.exception.ServerErrorCode.MODEL_METADATA_FILE_ERROR; +import static org.apache.kylin.common.exception.code.ErrorCodeServer.MODEL_ID_NOT_EXIST; +import static org.apache.kylin.common.exception.code.ErrorCodeServer.MODEL_NAME_DUPLICATE; +import static org.apache.kylin.common.exception.code.ErrorCodeServer.MODEL_NAME_INVALID; +import static org.apache.kylin.common.persistence.ResourceStore.METASTORE_UUID_TAG; +import static org.apache.kylin.common.persistence.ResourceStore.VERSION_FILE; +import static org.apache.kylin.metadata.model.schema.ImportModelContext.MODEL_REC_PATH; +import static org.apache.kylin.metadata.model.schema.SchemaNodeType.MODEL_DIM; +import static org.apache.kylin.metadata.model.schema.SchemaNodeType.MODEL_FACT; Review Comment: 格式不对？ > When there is a null value in the Kafka source data, the build job reports an > error > --- > > Key: KYLIN-5731 > URL: https://issues.apache.org/jira/browse/KYLIN-5731 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: zhong.zhu >Assignee: zhong.zhu >Priority: Minor > Fix For: 5.0.0 > > > If the field value in kafka json data is null, the task will report an error. > Null value field "clue_source_2_name":null > Field type is varchar > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798502#comment-17798502
]
ASF GitHub Bot commented on KYLIN-5731:
---
pfzhan commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1431123373
##
src/tool/src/main/java/org/apache/kylin/helper/MetadataToolHelper.java:
##
@@ -98,7 +97,7 @@ public void backup(KylinConfig kylinConfig, String dir,
String folder) throws Ex
}
public Pair backup(KylinConfig kylinConfig, String
project, String path, String folder,
-boolean compress, boolean excludeTableExd) throws Exception {
+ boolean compress, boolean
excludeTableExd) throws Exception {
Review Comment:
格式
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798500#comment-17798500
]
ASF GitHub Bot commented on KYLIN-5731:
---
pfzhan commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1431119481
##
src/core-job/src/main/java/org/apache/kylin/job/impl/threadpool/NDefaultScheduler.java:
##
@@ -232,6 +232,12 @@ public static double currentAvailableMem() {
return 1.0 * memoryRemaining.availablePermits();
}
+public int getMaxConcurrentJobLimitByProjectForInitThread(KylinConfig
config, JobEngineConfig jobEngineConfig,
+ String project)
{
+int jobLimit = getMaxConcurrentJobLimitByProject(config,
jobEngineConfig, project);
Review Comment:
format 不对
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798436#comment-17798436
]
ASF GitHub Bot commented on KYLIN-5731:
---
weiwei121723 commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1430942237
##
src/common-server/src/main/java/org/apache/kylin/rest/controller/NEpochController.java:
##
@@ -48,7 +51,11 @@ public class NEpochController extends NBasicController {
@PostMapping(value = "", produces = { HTTP_VND_APACHE_KYLIN_JSON,
HTTP_VND_APACHE_KYLIN_V4_PUBLIC_JSON })
@ResponseBody
public EnvelopeResponse updateEpochOwner(@RequestBody EpochRequest
epochRequest) {
-checkCollectionRequiredArg("projects", epochRequest.getProjects());
+if (Objects.isNull(epochRequest.getProjects())) {
+// Avoid following NPEs.
+epochRequest.setProjects(new ArrayList<>(0));
+}
+// Empty projects has specified meanings: all projects do change epoch.
Review Comment:
我主要担心这个接口，忘记传projects参数，不小心更新了所有的epoch。如果为空或者emptyList，应该直接报错比较稳妥些；
感觉/api/epoch/all 更适合做这个事情
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798419#comment-17798419
]
ASF GitHub Bot commented on KYLIN-5731:
---
thy950523 commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1430893586
##
src/common-server/src/main/java/org/apache/kylin/rest/controller/NEpochController.java:
##
@@ -48,7 +51,11 @@ public class NEpochController extends NBasicController {
@PostMapping(value = "", produces = { HTTP_VND_APACHE_KYLIN_JSON,
HTTP_VND_APACHE_KYLIN_V4_PUBLIC_JSON })
@ResponseBody
public EnvelopeResponse updateEpochOwner(@RequestBody EpochRequest
epochRequest) {
-checkCollectionRequiredArg("projects", epochRequest.getProjects());
+if (Objects.isNull(epochRequest.getProjects())) {
+// Avoid following NPEs.
+epochRequest.setProjects(new ArrayList<>(0));
+}
+// Empty projects has specified meanings: all projects do change epoch.
Review Comment:
这个issue是修复，projects参数为null情况下的NPE问题。
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798151#comment-17798151
]
ASF GitHub Bot commented on KYLIN-5731:
---
weiwei121723 commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1429932159
##
src/common-server/src/main/java/org/apache/kylin/rest/controller/NEpochController.java:
##
@@ -48,7 +51,11 @@ public class NEpochController extends NBasicController {
@PostMapping(value = "", produces = { HTTP_VND_APACHE_KYLIN_JSON,
HTTP_VND_APACHE_KYLIN_V4_PUBLIC_JSON })
@ResponseBody
public EnvelopeResponse updateEpochOwner(@RequestBody EpochRequest
epochRequest) {
-checkCollectionRequiredArg("projects", epochRequest.getProjects());
+if (Objects.isNull(epochRequest.getProjects())) {
+// Avoid following NPEs.
+epochRequest.setProjects(new ArrayList<>(0));
+}
+// Empty projects has specified meanings: all projects do change epoch.
Review Comment:
已经有一个updateAllEpochOwner接口了，这个接口默认更新所有的epoch，有点过重了；
就算要更新所有的epoch，也可以调用，org.apache.kylin.rest.service.EpochService#updateAllEpochs这个方法
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[ https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798146#comment-17798146 ] ASF GitHub Bot commented on KYLIN-5731: --- weiwei121723 commented on code in PR #2161: URL: https://github.com/apache/kylin/pull/2161#discussion_r1429997144 ## src/common-service/src/main/java/org/apache/kylin/rest/service/ProjectService.java: ## @@ -25,6 +25,7 @@ import static org.apache.kylin.common.constant.Constants.KYLIN_SOURCE_JDBC_SOURCE_ENABLE_KEY; import static org.apache.kylin.common.constant.Constants.KYLIN_SOURCE_JDBC_SOURCE_NAME_KEY; import static org.apache.kylin.common.constant.Constants.KYLIN_SOURCE_JDBC_USER_KEY; +import static org.apache.kylin.common.constant.Constants.KYLIN_JOB_MAX_CONCURRENT_JOBS; Review Comment: KYLIN-5738，这个commit的issue号写错了 > When there is a null value in the Kafka source data, the build job reports an > error > --- > > Key: KYLIN-5731 > URL: https://issues.apache.org/jira/browse/KYLIN-5731 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: zhong.zhu >Assignee: zhong.zhu >Priority: Minor > Fix For: 5.0.0 > > > If the field value in kafka json data is null, the task will report an error. > Null value field "clue_source_2_name":null > Field type is varchar > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798138#comment-17798138
]
ASF GitHub Bot commented on KYLIN-5731:
---
weiwei121723 commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1429954352
##
src/core-common/src/main/java/org/apache/kylin/common/msg/CnMessage.java:
##
@@ -32,6 +32,7 @@ public class CnMessage extends Message {
private static final String TASK_TIMEOUT = "执行超时";
private static final String PARAMETER_EMPTY = "请输入参数 “%s” 的值。";
+private static final String PARAMETER_MUST_BE_POSITIVE_NUMBER= "参数 %s
的值必须为非负数.";
Review Comment:
中文句号
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[
https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798134#comment-17798134
]
ASF GitHub Bot commented on KYLIN-5731:
---
weiwei121723 commented on code in PR #2161:
URL: https://github.com/apache/kylin/pull/2161#discussion_r1429932159
##
src/common-server/src/main/java/org/apache/kylin/rest/controller/NEpochController.java:
##
@@ -48,7 +51,11 @@ public class NEpochController extends NBasicController {
@PostMapping(value = "", produces = { HTTP_VND_APACHE_KYLIN_JSON,
HTTP_VND_APACHE_KYLIN_V4_PUBLIC_JSON })
@ResponseBody
public EnvelopeResponse updateEpochOwner(@RequestBody EpochRequest
epochRequest) {
-checkCollectionRequiredArg("projects", epochRequest.getProjects());
+if (Objects.isNull(epochRequest.getProjects())) {
+// Avoid following NPEs.
+epochRequest.setProjects(new ArrayList<>(0));
+}
+// Empty projects has specified meanings: all projects do change epoch.
Review Comment:
已经有一个updateAllEpochOwner接口了，这个接口默认更新所有的epoch，有点过重了
> When there is a null value in the Kafka source data, the build job reports an
> error
> ---
>
> Key: KYLIN-5731
> URL: https://issues.apache.org/jira/browse/KYLIN-5731
> Project: Kylin
> Issue Type: Bug
>Affects Versions: 5.0-beta
>Reporter: zhong.zhu
>Assignee: zhong.zhu
>Priority: Minor
> Fix For: 5.0.0
>
>
> If the field value in kafka json data is null, the task will report an error.
> Null value field "clue_source_2_name":null
> Field type is varchar
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[ https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796522#comment-17796522 ] ASF GitHub Bot commented on KYLIN-5731: --- thy950523 commented on code in PR #2161: URL: https://github.com/apache/kylin/pull/2161#discussion_r1426129574 ## src/tool/src/main/java/org/apache/kylin/helper/MetadataToolHelper.java: ## @@ -18,26 +18,8 @@ package org.apache.kylin.helper; -import static org.apache.kylin.common.exception.code.ErrorCodeTool.FILE_ALREADY_EXISTS; - -import java.io.File; -import java.io.IOException; -import java.net.URI; -import java.nio.file.FileSystems; -import java.nio.file.Paths; -import java.time.Clock; -import java.time.LocalDateTime; -import java.time.format.DateTimeFormatter; -import java.util.Collections; -import java.util.List; -import java.util.Locale; -import java.util.NavigableSet; -import java.util.Objects; -import java.util.Set; -import java.util.stream.Collectors; - -import javax.sql.DataSource; - +import lombok.val; Review Comment: 之前没有添加eclipse formatter的配置，重新commit了 > When there is a null value in the Kafka source data, the build job reports an > error > --- > > Key: KYLIN-5731 > URL: https://issues.apache.org/jira/browse/KYLIN-5731 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: zhong.zhu >Assignee: zhong.zhu >Priority: Minor > Fix For: 5.0.0 > > > If the field value in kafka json data is null, the task will report an error. > Null value field "clue_source_2_name":null > Field type is varchar > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[ https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796158#comment-17796158 ] ASF GitHub Bot commented on KYLIN-5731: --- hit-lacus commented on code in PR #2161: URL: https://github.com/apache/kylin/pull/2161#discussion_r1425193242 ## src/tool/src/main/java/org/apache/kylin/helper/MetadataToolHelper.java: ## @@ -18,26 +18,8 @@ package org.apache.kylin.helper; -import static org.apache.kylin.common.exception.code.ErrorCodeTool.FILE_ALREADY_EXISTS; - -import java.io.File; -import java.io.IOException; -import java.net.URI; -import java.nio.file.FileSystems; -import java.nio.file.Paths; -import java.time.Clock; -import java.time.LocalDateTime; -import java.time.format.DateTimeFormatter; -import java.util.Collections; -import java.util.List; -import java.util.Locale; -import java.util.NavigableSet; -import java.util.Objects; -import java.util.Set; -import java.util.stream.Collectors; - -import javax.sql.DataSource; - +import lombok.val; Review Comment: 这个 import 顺序对吗？ ## src/tool/src/main/java/org/apache/kylin/helper/MetadataToolHelper.java: ## @@ -18,26 +18,8 @@ package org.apache.kylin.helper; -import static org.apache.kylin.common.exception.code.ErrorCodeTool.FILE_ALREADY_EXISTS; - -import java.io.File; -import java.io.IOException; -import java.net.URI; -import java.nio.file.FileSystems; -import java.nio.file.Paths; -import java.time.Clock; -import java.time.LocalDateTime; -import java.time.format.DateTimeFormatter; -import java.util.Collections; -import java.util.List; -import java.util.Locale; -import java.util.NavigableSet; -import java.util.Objects; -import java.util.Set; -import java.util.stream.Collectors; - -import javax.sql.DataSource; - +import lombok.val; Review Comment: 这个 import 顺序对吗？ > When there is a null value in the Kafka source data, the build job reports an > error > --- > > Key: KYLIN-5731 > URL: https://issues.apache.org/jira/browse/KYLIN-5731 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: zhong.zhu >Assignee: zhong.zhu >Priority: Minor > Fix For: 5.0.0 > > > If the field value in kafka json data is null, the task will report an error. > Null value field "clue_source_2_name":null > Field type is varchar > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5731) When there is a null value in the Kafka source data, the build job reports an error
[ https://issues.apache.org/jira/browse/KYLIN-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795598#comment-17795598 ] ASF GitHub Bot commented on KYLIN-5731: --- thy950523 opened a new pull request, #2161: URL: https://github.com/apache/kylin/pull/2161 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit * [ ] Branch **kylin3** for v2.x to v3.x * [ ] Branch **kylin4** for v4.x * [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ * [x] Bugfix (non-breaking change which fixes an issue) * [ ] New feature (non-breaking change which adds functionality) * [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) * [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ * [x] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail * [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" * [x] Compiling and unit tests pass locally with my changes * [x] I have added tests that prove my fix is effective or that my feature works * [x] I have added necessary documentation (if appropriate) * [x] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at [u...@kylin.apache.org](mailto:u...@kylin.apache.org) or [d...@kylin.apache.org](mailto:d...@kylin.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... > When there is a null value in the Kafka source data, the build job reports an > error > --- > > Key: KYLIN-5731 > URL: https://issues.apache.org/jira/browse/KYLIN-5731 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: zhong.zhu >Assignee: zhong.zhu >Priority: Minor > Fix For: 5.0.0 > > > If the field value in kafka json data is null, the task will report an error. > Null value field "clue_source_2_name":null > Field type is varchar > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5704) For ‘in’ condition query of non-time partition columns, when the data type of the value in 'in' condition is inconsistent with that of the non-time partition column, th
[ https://issues.apache.org/jira/browse/KYLIN-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788274#comment-17788274 ] ASF GitHub Bot commented on KYLIN-5704: --- hit-lacus merged PR #2159: URL: https://github.com/apache/kylin/pull/2159 > For ‘in’ condition query of non-time partition columns, when the data type of > the value in 'in' condition is inconsistent with that of the non-time > partition column, the segment pruner fails, resulting in full Segment scanning > -- > > Key: KYLIN-5704 > URL: https://issues.apache.org/jira/browse/KYLIN-5704 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: Hongrong Cao >Assignee: Guangyuan Feng >Priority: Major > Fix For: 5.0.0 > > > The query column is a non-time partition column, a common dimension column, > and the filter condition of the common dimension column is col in (x1, x2...) > In this case (and because the col and x1 types do not match, it is > automatically converted to (cast col as string) in (x1,x2..), Fileprunner > will report an error because > org.apache.spark.sql.execution.datasource.FilePruner#convertCastFilter does > not handle in. > Explain that the convertCastFilter method is to remove the cast condition, so > that the filter condition can be matched when calling > DataSourceStrategy.translateFilter, and then the Segment can be filtered. > However, currently convertCastFilter misses the processing of the in > condition, so translateFilter cannot match and becomes empty, so The query > was thrown incorrectly. > In addition: if it is a time partition column, it does not matter if an error > is reported here, because in the previous steps, the calcite file prunner has > already completed the Segment Prune of the time partition column. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5704) For ‘in’ condition query of non-time partition columns, when the data type of the value in 'in' condition is inconsistent with that of the non-time partition column, th
[ https://issues.apache.org/jira/browse/KYLIN-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787783#comment-17787783 ] ASF GitHub Bot commented on KYLIN-5704: --- Aligadodo closed pull request #2158: KYLIN-5704 (run ci test only) URL: https://github.com/apache/kylin/pull/2158 > For ‘in’ condition query of non-time partition columns, when the data type of > the value in 'in' condition is inconsistent with that of the non-time > partition column, the segment pruner fails, resulting in full Segment scanning > -- > > Key: KYLIN-5704 > URL: https://issues.apache.org/jira/browse/KYLIN-5704 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: Hongrong Cao >Assignee: Guangyuan Feng >Priority: Major > Fix For: 5.0.0 > > > The query column is a non-time partition column, a common dimension column, > and the filter condition of the common dimension column is col in (x1, x2...) > In this case (and because the col and x1 types do not match, it is > automatically converted to (cast col as string) in (x1,x2..), Fileprunner > will report an error because > org.apache.spark.sql.execution.datasource.FilePruner#convertCastFilter does > not handle in. > Explain that the convertCastFilter method is to remove the cast condition, so > that the filter condition can be matched when calling > DataSourceStrategy.translateFilter, and then the Segment can be filtered. > However, currently convertCastFilter misses the processing of the in > condition, so translateFilter cannot match and becomes empty, so The query > was thrown incorrectly. > In addition: if it is a time partition column, it does not matter if an error > is reported here, because in the previous steps, the calcite file prunner has > already completed the Segment Prune of the time partition column. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5704) For ‘in’ condition query of non-time partition columns, when the data type of the value in 'in' condition is inconsistent with that of the non-time partition column, th
[ https://issues.apache.org/jira/browse/KYLIN-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786251#comment-17786251 ] ASF GitHub Bot commented on KYLIN-5704: --- Aligadodo opened a new pull request, #2158: URL: https://github.com/apache/kylin/pull/2158 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > For ‘in’ condition query of non-time partition columns, when the data type of > the value in 'in' condition is inconsistent with that of the non-time > partition column, the segment pruner fails, resulting in full Segment scanning > -- > > Key: KYLIN-5704 > URL: https://issues.apache.org/jira/browse/KYLIN-5704 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-beta >Reporter: Hongrong Cao >Assignee: Guangyuan Feng >Priority: Major > Fix For: 5.0.0 > > > The query column is a non-time partition column, a common dimension column, > and the filter condition of the common dimension column is col in (x1, x2...) > In this case (and because the col and x1 types do not match, it is > automatically converted to (cast col as string) in (x1,x2..), Fileprunner > will report an error because > org.apache.spark.sql.execution.datasource.FilePruner#convertCastFilter does > not handle in. > Explain that the convertCastFilter method is to remove the cast condition, so > that the filter condition can be matched when calling > DataSourceStrategy.translateFilter, and then the Segment can be filtered. > However, currently convertCastFilter misses the processing of the in > condition, so translateFilter cannot match and becomes empty, so The query > was thrown incorrectly. > In addition: if it is a time partition column, it does not matter if an error > is reported here, because in the previous steps, the calcite file prunner has > already completed the Segment Prune of the time partition column. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5633) The query can answered with the existing data of the current model (accepting index or segment data is not uniform), and there should be no query failure or push-down
[ https://issues.apache.org/jira/browse/KYLIN-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756698#comment-17756698 ] ASF GitHub Bot commented on KYLIN-5633: --- ygjia opened a new pull request, #2148: URL: https://github.com/apache/kylin/pull/2148 ## Proposed changes issue list https://issues.apache.org/jira/browse/KYLIN-5633 https://issues.apache.org/jira/browse/KYLIN-5687 https://issues.apache.org/jira/browse/KYLIN-5690 https://issues.apache.org/jira/browse/KYLIN-5691 https://issues.apache.org/jira/browse/KYLIN-5692 https://issues.apache.org/jira/browse/KYLIN-5693 https://issues.apache.org/jira/browse/KYLIN-5694 https://issues.apache.org/jira/browse/KYLIN-5695 https://issues.apache.org/jira/browse/KYLIN-5696 https://issues.apache.org/jira/browse/KYLIN-5697 https://issues.apache.org/jira/browse/KYLIN-5698 https://issues.apache.org/jira/browse/KYLIN-5699 https://issues.apache.org/jira/browse/KYLIN-5700 Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have added necessary documentation (if appropriate) - [x] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > The query can answered with the existing data of the current model (accepting > index or segment data is not uniform), and there should be no query failure > or push-down > -- > > Key: KYLIN-5633 > URL: https://issues.apache.org/jira/browse/KYLIN-5633 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: 5.0-alpha >Reporter: Zhiting Guo >Assignee: Zhiting Guo >Priority: Major > Fix For: 5.0-beta > > Attachments: Segment heterogeneous query behavior (1).pdf > > > Details are included in the attachment. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5474) Kylin5 throw [null Exception] org.h2.Driver in local debug mode
[ https://issues.apache.org/jira/browse/KYLIN-5474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754423#comment-17754423 ] ASF GitHub Bot commented on KYLIN-5474: --- ygjia closed pull request #2101: KYLIN-5474 fix kylin5 local debug mode URL: https://github.com/apache/kylin/pull/2101 > Kylin5 throw [null Exception] org.h2.Driver in local debug mode > - > > Key: KYLIN-5474 > URL: https://issues.apache.org/jira/browse/KYLIN-5474 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Minor > Fix For: 5.0-beta > > Attachments: image (1).png, image (2).png > > > For Kyln5 local debug mode, please refer -> > https://issues.apache.org/jira/browse/KYLIN-5269 > > kylin5 throw [null Exception] org.h2.Driver when run sql in local debug mode > !image (1).png|width=1017,height=417! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5686) query result open ModelAggregate and IndexDetails issue
[ https://issues.apache.org/jira/browse/KYLIN-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753016#comment-17753016 ] ASF GitHub Bot commented on KYLIN-5686: --- hit-lacus merged PR #2147: URL: https://github.com/apache/kylin/pull/2147 > query result open ModelAggregate and IndexDetails issue > --- > > Key: KYLIN-5686 > URL: https://issues.apache.org/jira/browse/KYLIN-5686 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5686) query result open ModelAggregate and IndexDetails issue
[ https://issues.apache.org/jira/browse/KYLIN-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752720#comment-17752720 ] ASF GitHub Bot commented on KYLIN-5686: --- LauraXia123 opened a new pull request, #2147: URL: https://github.com/apache/kylin/pull/2147 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > query result open ModelAggregate and IndexDetails issue > --- > > Key: KYLIN-5686 > URL: https://issues.apache.org/jira/browse/KYLIN-5686 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5685) query result empty page issue
[ https://issues.apache.org/jira/browse/KYLIN-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752653#comment-17752653 ] ASF GitHub Bot commented on KYLIN-5685: --- hit-lacus merged PR #2146: URL: https://github.com/apache/kylin/pull/2146 > query result empty page issue > - > > Key: KYLIN-5685 > URL: https://issues.apache.org/jira/browse/KYLIN-5685 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5685) query result empty page issue
[ https://issues.apache.org/jira/browse/KYLIN-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752609#comment-17752609 ] ASF GitHub Bot commented on KYLIN-5685: --- LauraXia123 opened a new pull request, #2146: URL: https://github.com/apache/kylin/pull/2146 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > query result empty page issue > - > > Key: KYLIN-5685 > URL: https://issues.apache.org/jira/browse/KYLIN-5685 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5681) SCD2 can not work
[ https://issues.apache.org/jira/browse/KYLIN-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752285#comment-17752285 ] ASF GitHub Bot commented on KYLIN-5681: --- hit-lacus merged PR #2145: URL: https://github.com/apache/kylin/pull/2145 > SCD2 can not work > -- > > Key: KYLIN-5681 > URL: https://issues.apache.org/jira/browse/KYLIN-5681 > Project: Kylin > Issue Type: Bug > Components: Modeling >Affects Versions: 5.0-alpha >Reporter: pengfei.zhan >Assignee: pengfei.zhan >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-08-09-13-12-20-353.png > > > [https://kylin.apache.org/5.0/docs/modeling/model_design/slowly_changing_dimension#scd2-model-based-on-history-table] > I create a scd2 model, but get an error message. > !image-2023-08-09-13-12-20-353.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5676) If the query matches an index that is not built, the front-end prompts you
[ https://issues.apache.org/jira/browse/KYLIN-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752271#comment-17752271 ] ASF GitHub Bot commented on KYLIN-5676: --- hit-lacus merged PR #2144: URL: https://github.com/apache/kylin/pull/2144 > If the query matches an index that is not built, the front-end prompts you > -- > > Key: KYLIN-5676 > URL: https://issues.apache.org/jira/browse/KYLIN-5676 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5681) SCD2 can not work
[ https://issues.apache.org/jira/browse/KYLIN-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752267#comment-17752267 ] ASF GitHub Bot commented on KYLIN-5681: --- pfzhan commented on PR #2145: URL: https://github.com/apache/kylin/pull/2145#issuecomment-1670701765 retest this please > SCD2 can not work > -- > > Key: KYLIN-5681 > URL: https://issues.apache.org/jira/browse/KYLIN-5681 > Project: Kylin > Issue Type: Bug > Components: Modeling >Affects Versions: 5.0-alpha >Reporter: pengfei.zhan >Assignee: pengfei.zhan >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-08-09-13-12-20-353.png > > > [https://kylin.apache.org/5.0/docs/modeling/model_design/slowly_changing_dimension#scd2-model-based-on-history-table] > I create a scd2 model, but get an error message. > !image-2023-08-09-13-12-20-353.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5681) SCD2 can not work
[ https://issues.apache.org/jira/browse/KYLIN-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752263#comment-17752263 ] ASF GitHub Bot commented on KYLIN-5681: --- pfzhan opened a new pull request, #2145: URL: https://github.com/apache/kylin/pull/2145 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > SCD2 can not work > -- > > Key: KYLIN-5681 > URL: https://issues.apache.org/jira/browse/KYLIN-5681 > Project: Kylin > Issue Type: Bug > Components: Modeling >Affects Versions: 5.0-alpha >Reporter: pengfei.zhan >Assignee: pengfei.zhan >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-08-09-13-12-20-353.png > > > [https://kylin.apache.org/5.0/docs/modeling/model_design/slowly_changing_dimension#scd2-model-based-on-history-table] > I create a scd2 model, but get an error message. > !image-2023-08-09-13-12-20-353.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5676) If the query matches an index that is not built, the front-end prompts you
[
https://issues.apache.org/jira/browse/KYLIN-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752233#comment-17752233
]
ASF GitHub Bot commented on KYLIN-5676:
---
hit-lacus commented on code in PR #2144:
URL: https://github.com/apache/kylin/pull/2144#discussion_r1287864339
##
kystudio/src/components/studio/StudioModel/TableIndexEdit/locales.js:
##
@@ -22,14 +23,27 @@ export default {
moveDown: 'Move down',
tableIndexShardByTips: 'Please select columns for detail query. To enhance
the query performance, please move the frequently used dimensions to the top of
the list, and set a column with relatively large cardinality as ShardBy.',
filterByColumns: 'Search by column name',
-excludeTableCheckbox: 'Display columns excluded from recommendations',
-excludeTableCheckboxTip: 'If an excluded column is added to indexes, this
column will store "historical truth"(SCD Type2, As Was).If you want to use
"latest status" (SCD Type 1, As Is) ,please don\'t add this column to any index
and add foreign key instead.',
-excludedTableIconTip: 'Excluded from recommendations',
-manyToManyAntiTableTip: 'For the tables excluded from recommendations, if
the join relationship of a table is One-to-Many or Many-to-Many, dimensions
from this table can\'t be used in indexes. ',
-indexTimeRange: 'Index’s Time Range',
-indexTimeRangeTips: 'The data range that the indexes will be built in.
With "Batch and Streaming" selected, there will be generated batch indexes and
streaming indexes with same content respectively. ',
+excludeTableCheckbox: 'Display columns excluded',
+excludeTableCheckboxTip1: 'Exclude columns using the slow-changing
dimension to record data changes:',
+excludeTableCheckboxTip2: 'When you want the column to reflect historical
facts (SCD Type2, As Was), please check this option and add it to the index.',
+excludeTableCheckboxTip3: 'When you want the column to reflect the latest
state of the dimension (SCD Type1, As Is), do not check this option. Instead,
add a foreign key to the index.',
+excludeTableCheckboxTip4: 'Not familiar with the slow-changing
dimension? https://docs.kyligence.io/books/v4.6/en/Designers-Guide/model/model_design/slowly_changing_dimension.en.html?h=Slowly%20Changing%20Dimension;
target="_blank">Check the manual',
+manyToManyAntiTableTip: 'The table columns excluded, if the join
relationship of a table is One-to-Many or Many-to-Many, dimensions from this
table can\'t be used in indexes. ',
+indexTimeRange: 'Real-time Index Types',
+indexTimeRangeTips: 'After selection, the table index will generate
streaming indexes and batch indexes of the same content respectively.',
noIndexRangeByHybrid: 'Select index’s data range to display available
columns.',
textRecognition: 'Text Recognition',
-refuseAddIndexTip: 'Can\'t add streaming indexes. Please stop the
streaming job and then delete all the streaming segments.'
+refuseAddIndexTip: 'Can\'t add streaming indexes. Please stop the
streaming job and then delete all the streaming segments.',
+sugessionLabel1: 'Rec.1',
+sugession1: 'Sort by common usage frequency. Reasonable ordering can
significantly improve the efficiency of table queries.',
+sugessionLabel2: 'Rec.2',
+sugession2: 'Set one high-cardinality column as shardby. The shardby
column is used to store data in slices, which can avoid uneven dispersion of
table data, improve parallelism and query efficiency.',
+tips: 'No cardinality information? Please sample first and try again. ',
+goToDataSource: 'Go to sampling',
+knowMore: 'For more information, please ',
+userManual: 'view the manual',
+shardbyManal:
'https://docs.kyligence.io/books/v4.6/en/Designers-Guide/model/model_design/table_index.en.html?q=shardBy',
Review Comment:
This url is not right.
> If the query matches an index that is not built, the front-end prompts you
> --
>
> Key: KYLIN-5676
> URL: https://issues.apache.org/jira/browse/KYLIN-5676
> Project: Kylin
> Issue Type: Bug
>Reporter: Laura Xia
>Assignee: Laura Xia
>Priority: Major
> Fix For: 5.0-beta
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5676) If the query matches an index that is not built, the front-end prompts you
[ https://issues.apache.org/jira/browse/KYLIN-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751000#comment-17751000 ] ASF GitHub Bot commented on KYLIN-5676: --- LauraXia123 opened a new pull request, #2144: URL: https://github.com/apache/kylin/pull/2144 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > If the query matches an index that is not built, the front-end prompts you > -- > > Key: KYLIN-5676 > URL: https://issues.apache.org/jira/browse/KYLIN-5676 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5668) The jump page after editing the model and discarding the edit is incorrect
[ https://issues.apache.org/jira/browse/KYLIN-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750602#comment-17750602 ] ASF GitHub Bot commented on KYLIN-5668: --- hit-lacus merged PR #2142: URL: https://github.com/apache/kylin/pull/2142 > The jump page after editing the model and discarding the edit is incorrect > -- > > Key: KYLIN-5668 > URL: https://issues.apache.org/jira/browse/KYLIN-5668 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5668) The jump page after editing the model and discarding the edit is incorrect
[ https://issues.apache.org/jira/browse/KYLIN-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748553#comment-17748553 ] ASF GitHub Bot commented on KYLIN-5668: --- LauraXia123 opened a new pull request, #2142: URL: https://github.com/apache/kylin/pull/2142 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > The jump page after editing the model and discarding the edit is incorrect > -- > > Key: KYLIN-5668 > URL: https://issues.apache.org/jira/browse/KYLIN-5668 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5667) Model page white screen issue
[ https://issues.apache.org/jira/browse/KYLIN-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747280#comment-17747280 ] ASF GitHub Bot commented on KYLIN-5667: --- hit-lacus merged PR #2141: URL: https://github.com/apache/kylin/pull/2141 > Model page white screen issue > - > > Key: KYLIN-5667 > URL: https://issues.apache.org/jira/browse/KYLIN-5667 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5667) Model page white screen issue
[ https://issues.apache.org/jira/browse/KYLIN-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747247#comment-17747247 ] ASF GitHub Bot commented on KYLIN-5667: --- LauraXia123 opened a new pull request, #2141: URL: https://github.com/apache/kylin/pull/2141 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Model page white screen issue > - > > Key: KYLIN-5667 > URL: https://issues.apache.org/jira/browse/KYLIN-5667 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5654) Diagnostic package API (/kylin/api/system/diag? host= ip:port) The current design has security risks
[ https://issues.apache.org/jira/browse/KYLIN-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746358#comment-17746358 ] ASF GitHub Bot commented on KYLIN-5654: --- hit-lacus merged PR #2139: URL: https://github.com/apache/kylin/pull/2139 > Diagnostic package API (/kylin/api/system/diag? host= ip:port) The current > design has security risks > > > Key: KYLIN-5654 > URL: https://issues.apache.org/jira/browse/KYLIN-5654 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > > RT -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5654) Diagnostic package API (/kylin/api/system/diag? host= ip:port) The current design has security risks
[ https://issues.apache.org/jira/browse/KYLIN-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745525#comment-17745525 ] ASF GitHub Bot commented on KYLIN-5654: --- LauraXia123 opened a new pull request, #2139: URL: https://github.com/apache/kylin/pull/2139 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Diagnostic package API (/kylin/api/system/diag? host= ip:port) The current > design has security risks > > > Key: KYLIN-5654 > URL: https://issues.apache.org/jira/browse/KYLIN-5654 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > > RT -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5618) Unify the copywriting of the ShardBy column on the Web GUI
[ https://issues.apache.org/jira/browse/KYLIN-5618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743736#comment-17743736 ] ASF GitHub Bot commented on KYLIN-5618: --- hit-lacus merged PR #2136: URL: https://github.com/apache/kylin/pull/2136 > Unify the copywriting of the ShardBy column on the Web GUI > -- > > Key: KYLIN-5618 > URL: https://issues.apache.org/jira/browse/KYLIN-5618 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-07-12-11-35-54-602.png, > image-2023-07-12-11-36-02-682.png > > > !image-2023-07-12-11-35-54-602.png! > !image-2023-07-12-11-36-02-682.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5618) Unify the copywriting of the ShardBy column on the Web GUI
[ https://issues.apache.org/jira/browse/KYLIN-5618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742714#comment-17742714 ] ASF GitHub Bot commented on KYLIN-5618: --- LauraXia123 opened a new pull request, #2136: URL: https://github.com/apache/kylin/pull/2136 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Unify the copywriting of the ShardBy column on the Web GUI > -- > > Key: KYLIN-5618 > URL: https://issues.apache.org/jira/browse/KYLIN-5618 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-07-12-11-35-54-602.png, > image-2023-07-12-11-36-02-682.png > > > !image-2023-07-12-11-35-54-602.png! > !image-2023-07-12-11-36-02-682.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5589) Supports semi-additive measure
[ https://issues.apache.org/jira/browse/KYLIN-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740425#comment-17740425 ] ASF GitHub Bot commented on KYLIN-5589: --- hit-lacus merged PR #2134: URL: https://github.com/apache/kylin/pull/2134 > Supports semi-additive measure > -- > > Key: KYLIN-5589 > URL: https://issues.apache.org/jira/browse/KYLIN-5589 > Project: Kylin > Issue Type: New Feature >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > > 创建模型时，增加半累加度量，同时在模型中正确保存半累加度量的元数据。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5597) When the /etc/hosts file of the Kylin node does not configure the IP corresponding to the cluster domain name in the hadoop conf configuration file, Kylin will fail to
[
https://issues.apache.org/jira/browse/KYLIN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740404#comment-17740404
]
ASF GitHub Bot commented on KYLIN-5597:
---
hit-lacus merged PR #2132:
URL: https://github.com/apache/kylin/pull/2132
> When the /etc/hosts file of the Kylin node does not configure the IP
> corresponding to the cluster domain name in the hadoop conf configuration
> file, Kylin will fail to start
> -
>
> Key: KYLIN-5597
> URL: https://issues.apache.org/jira/browse/KYLIN-5597
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Security
>Affects Versions: 5.0-alpha
>Reporter: huangsheng
>Assignee: huangsheng
>Priority: Major
> Fix For: 5.0-beta
>
>
> If the IP corresponding to the cluster domain name in the hadoop conf
> configuration file is not configured in the /etc/hosts file of the newly
> added KE node, the command
> {code:java}
> get-properties.sh kylin.kerberos.enabled{code}
> will fail when the service starts, and throw java.net.UnknownHostException
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5589) Supports semi-additive measure
[ https://issues.apache.org/jira/browse/KYLIN-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740154#comment-17740154 ] ASF GitHub Bot commented on KYLIN-5589: --- LauraXia123 opened a new pull request, #2134: URL: https://github.com/apache/kylin/pull/2134 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Supports semi-additive measure > -- > > Key: KYLIN-5589 > URL: https://issues.apache.org/jira/browse/KYLIN-5589 > Project: Kylin > Issue Type: New Feature >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > > 创建模型时，增加半累加度量，同时在模型中正确保存半累加度量的元数据。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5604) Open API for adding aggregation group function
[ https://issues.apache.org/jira/browse/KYLIN-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740044#comment-17740044 ] ASF GitHub Bot commented on KYLIN-5604: --- hit-lacus merged PR #2133: URL: https://github.com/apache/kylin/pull/2133 > Open API for adding aggregation group function > -- > > Key: KYLIN-5604 > URL: https://issues.apache.org/jira/browse/KYLIN-5604 > Project: Kylin > Issue Type: New Feature > Components: REST Service >Affects Versions: 5.0-alpha >Reporter: huangsheng >Assignee: huangsheng >Priority: Major > Fix For: 5.0-beta > > > Since the latest version of Kylin does not open API interfaces for functions > such as model editing, task management-task deletion, adding detailed > indexes, and adding aggregation groups, etc. > We are currently working on an indicator platform, and we need to use these > interfaces to nest into our own programs for secondary development, so we > hope to open up these APIs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5604) Open API for adding aggregation group function
[
https://issues.apache.org/jira/browse/KYLIN-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740006#comment-17740006
]
ASF GitHub Bot commented on KYLIN-5604:
---
Mrhs121 commented on code in PR #2133:
URL: https://github.com/apache/kylin/pull/2133#discussion_r1252478543
##
website/docs/restapi/model_api/model_management_api.md:
##
@@ -1178,3 +1178,92 @@ curl -X GET \
```
+### Add aggregate groups
+
+- `PUT http://host:port/kylin/api/index_plans/agg_groups`
+
+- Introduced in: 5.0
+
+- HTTP Header
+
+ - `Accept: application/vnd.apache.kylin-v4-public+json`
+ - `Accept-Language: en`
+ - `Content-Type: application/json;charset=utf-8`
+
+- HTT`P Body: JSON Object`
Review Comment:
修改好了
##
website/docs/restapi/model_api/model_management_api.md:
##
@@ -1178,3 +1178,92 @@ curl -X GET \
```
+### Add aggregate groups
+
+- `PUT http://host:port/kylin/api/index_plans/agg_groups`
+
+- Introduced in: 5.0
+
+- HTTP Header
+
+ - `Accept: application/vnd.apache.kylin-v4-public+json`
+ - `Accept-Language: en`
+ - `Content-Type: application/json;charset=utf-8`
+
+- HTT`P Body: JSON Object`
+
+ - `project` - `required` `string`, project name
+ - `model` - `required` `string`, model name
+ - `aggregation_groups` - `required` `JSON Object[]`, agg group json array,
can add multiple agg groups
+- `dimensions` - `required` `array[string]`, dimension array, must be in
database.table format, and the added dimension must have been added as a
dimension in the model
+- `measures` - `optional` `array[string]`, measure array, the added
measure name is the measure name defined in the model, case-sensitive
+- `mandatory_dims` - `optional` `array[string]`, mandatory dimension
array, the dimensions that must be included in the index generated by the
aggregate group
+- `hierarchy_dims` - `optional` `array[array[string]]`, hierarchy
dimension array, the dimensions that can be optimized according to the
hierarchical relationship in the index generated by the aggregate group, a set
of hierarchy dimensions in an array
+- `joint_dims` - `optional` `array[array[string]]`, joint dimension array,
the dimensions that must also be included in the index generated by the
aggregate group, a set of joint dimensions in an array
+> **NOTICE**： `mandatory_dims` 、`hierarchy_dims` 、`joint_dims` must be in
`dimensions`, and any dimension can only be set once in mandatory dimension,
hierarchy dimension or joint dimension.
+- `dim_cap` - `optional` `int`, Maximum number of dimension
combinations(MDC) for a single aggregate group, positive integer
+ - `global_dim_cap` - `optional` `int`, global MDC in this request, priority
global_dim_cap < dim_cap
+ - `restore_deleted_index` - `optional` `boolean`, When the indexes generated
by the aggregate group have been deleted, whether to generate these indexes
again when adding the aggregate group, the default is false, which means not to
add.
+
+- Curl Request Example
+
+ ```sh
+ curl -X PUT \
+ 'http://localhost:7070/kylin/api/index_plans/agg_groups' \
+ -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
+ -H 'Accept-Language: cn' \
+ -H 'Authorization: Basic QURNSU46S1lMSU4=' \
+ -H 'Content-Type: application/json;charset=utf-8'
+ -d '{"project":"ssb",
+ "model":"testNewName",
+ "aggregation_groups":[
+ {
+"dimensions":[
+"CUSTOMER_DETAILS.IMei",
+"CUSTOMER_DETAILS.REGION",
+"CUSTOMER_DETAILS.BALANCE"
+],
+"measures":[
+"ME2"
+],
+"mandatory_dims":[
+"CUSTOMER_DETAILS.reGION"
+],
+"hierarchy_dims":[
+[
+"CUSTOMER_DETAILS.Imei"
+]
+],
+ "joint_dims":[
+[
+"CUSTOMER_DETAILS.BALANCE"
+]
+],
+"dim_cap": 4
+ }
+ ],
+ "global_dim_cap":3,
+ "restore_deleted_index":false
+ }'
+ ```
+
+- Response Details
+ - code - string, response code, succeed: 000, failed: 999
+ - removed_layouts_size - The number of indexes to delete
+ - added_layouts_size - The number of indexes to add
+ - recovered_layouts_size - The number of deleted indexes in the generated
indexes, when restore_deleted_index = true, these indexes will be added
+
+- Response Details
Review Comment:
修改好了
> Open API for adding aggregation group function
> --
>
> Key: KYLIN-5604
> URL: https://issues.apache.org/jira/browse/KYLIN-5604
> Project: Kylin
> Issue Type: New Feature
> Components: REST Service
>Affects Versions: 5.0-alpha
>Reporter: huangsheng
>Assignee:
[jira] [Commented] (KYLIN-5604) Open API for adding aggregation group function
[ https://issues.apache.org/jira/browse/KYLIN-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740003#comment-17740003 ] ASF GitHub Bot commented on KYLIN-5604: --- hit-lacus commented on code in PR #2133: URL: https://github.com/apache/kylin/pull/2133#discussion_r1252474526 ## website/docs/restapi/model_api/model_management_api.md: ## @@ -1178,3 +1178,92 @@ curl -X GET \ ``` +### Add aggregate groups + +- `PUT http://host:port/kylin/api/index_plans/agg_groups` + +- Introduced in: 5.0 + +- HTTP Header + + - `Accept: application/vnd.apache.kylin-v4-public+json` + - `Accept-Language: en` + - `Content-Type: application/json;charset=utf-8` + +- HTT`P Body: JSON Object` Review Comment: 是不是不需要添加反斜线 backtick > Open API for adding aggregation group function > -- > > Key: KYLIN-5604 > URL: https://issues.apache.org/jira/browse/KYLIN-5604 > Project: Kylin > Issue Type: New Feature > Components: REST Service >Affects Versions: 5.0-alpha >Reporter: huangsheng >Assignee: huangsheng >Priority: Major > Fix For: 5.0-beta > > > Since the latest version of Kylin does not open API interfaces for functions > such as model editing, task management-task deletion, adding detailed > indexes, and adding aggregation groups, etc. > We are currently working on an indicator platform, and we need to use these > interfaces to nest into our own programs for secondary development, so we > hope to open up these APIs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5604) Open API for adding aggregation group function
[
https://issues.apache.org/jira/browse/KYLIN-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740002#comment-17740002
]
ASF GitHub Bot commented on KYLIN-5604:
---
hit-lacus commented on code in PR #2133:
URL: https://github.com/apache/kylin/pull/2133#discussion_r1252474350
##
website/docs/restapi/model_api/model_management_api.md:
##
@@ -1178,3 +1178,92 @@ curl -X GET \
```
+### Add aggregate groups
+
+- `PUT http://host:port/kylin/api/index_plans/agg_groups`
+
+- Introduced in: 5.0
+
+- HTTP Header
+
+ - `Accept: application/vnd.apache.kylin-v4-public+json`
+ - `Accept-Language: en`
+ - `Content-Type: application/json;charset=utf-8`
+
+- HTT`P Body: JSON Object`
+
+ - `project` - `required` `string`, project name
+ - `model` - `required` `string`, model name
+ - `aggregation_groups` - `required` `JSON Object[]`, agg group json array,
can add multiple agg groups
+- `dimensions` - `required` `array[string]`, dimension array, must be in
database.table format, and the added dimension must have been added as a
dimension in the model
+- `measures` - `optional` `array[string]`, measure array, the added
measure name is the measure name defined in the model, case-sensitive
+- `mandatory_dims` - `optional` `array[string]`, mandatory dimension
array, the dimensions that must be included in the index generated by the
aggregate group
+- `hierarchy_dims` - `optional` `array[array[string]]`, hierarchy
dimension array, the dimensions that can be optimized according to the
hierarchical relationship in the index generated by the aggregate group, a set
of hierarchy dimensions in an array
+- `joint_dims` - `optional` `array[array[string]]`, joint dimension array,
the dimensions that must also be included in the index generated by the
aggregate group, a set of joint dimensions in an array
+> **NOTICE**： `mandatory_dims` 、`hierarchy_dims` 、`joint_dims` must be in
`dimensions`, and any dimension can only be set once in mandatory dimension,
hierarchy dimension or joint dimension.
+- `dim_cap` - `optional` `int`, Maximum number of dimension
combinations(MDC) for a single aggregate group, positive integer
+ - `global_dim_cap` - `optional` `int`, global MDC in this request, priority
global_dim_cap < dim_cap
+ - `restore_deleted_index` - `optional` `boolean`, When the indexes generated
by the aggregate group have been deleted, whether to generate these indexes
again when adding the aggregate group, the default is false, which means not to
add.
+
+- Curl Request Example
+
+ ```sh
+ curl -X PUT \
+ 'http://localhost:7070/kylin/api/index_plans/agg_groups' \
+ -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
+ -H 'Accept-Language: cn' \
+ -H 'Authorization: Basic QURNSU46S1lMSU4=' \
+ -H 'Content-Type: application/json;charset=utf-8'
+ -d '{"project":"ssb",
+ "model":"testNewName",
+ "aggregation_groups":[
+ {
+"dimensions":[
+"CUSTOMER_DETAILS.IMei",
+"CUSTOMER_DETAILS.REGION",
+"CUSTOMER_DETAILS.BALANCE"
+],
+"measures":[
+"ME2"
+],
+"mandatory_dims":[
+"CUSTOMER_DETAILS.reGION"
+],
+"hierarchy_dims":[
+[
+"CUSTOMER_DETAILS.Imei"
+]
+],
+ "joint_dims":[
+[
+"CUSTOMER_DETAILS.BALANCE"
+]
+],
+"dim_cap": 4
+ }
+ ],
+ "global_dim_cap":3,
+ "restore_deleted_index":false
+ }'
+ ```
+
+- Response Details
+ - code - string, response code, succeed: 000, failed: 999
+ - removed_layouts_size - The number of indexes to delete
+ - added_layouts_size - The number of indexes to add
+ - recovered_layouts_size - The number of deleted indexes in the generated
indexes, when restore_deleted_index = true, these indexes will be added
+
+- Response Details
Review Comment:
Response Details or Response Example？
> Open API for adding aggregation group function
> --
>
> Key: KYLIN-5604
> URL: https://issues.apache.org/jira/browse/KYLIN-5604
> Project: Kylin
> Issue Type: New Feature
> Components: REST Service
>Affects Versions: 5.0-alpha
>Reporter: huangsheng
>Assignee: huangsheng
>Priority: Major
> Fix For: 5.0-beta
>
>
> Since the latest version of Kylin does not open API interfaces for functions
> such as model editing, task management-task deletion, adding detailed
> indexes, and adding aggregation groups, etc.
> We are currently working on an indicator platform, and we need to use these
> interfaces to nest into our own programs for secondary
[jira] [Commented] (KYLIN-5604) Open API for adding aggregation group function
[ https://issues.apache.org/jira/browse/KYLIN-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739873#comment-17739873 ] ASF GitHub Bot commented on KYLIN-5604: --- Mrhs121 opened a new pull request, #2133: URL: https://github.com/apache/kylin/pull/2133 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Open API for adding aggregation group function > -- > > Key: KYLIN-5604 > URL: https://issues.apache.org/jira/browse/KYLIN-5604 > Project: Kylin > Issue Type: New Feature > Components: REST Service >Affects Versions: 5.0-alpha >Reporter: huangsheng >Assignee: huangsheng >Priority: Major > Fix For: 5.0-beta > > > Since the latest version of Kylin does not open API interfaces for functions > such as model editing, task management-task deletion, adding detailed > indexes, and adding aggregation groups, etc. > We are currently working on an indicator platform, and we need to use these > interfaces to nest into our own programs for secondary development, so we > hope to open up these APIs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5597) When the /etc/hosts file of the Kylin node does not configure the IP corresponding to the cluster domain name in the hadoop conf configuration file, Kylin will fail to
[
https://issues.apache.org/jira/browse/KYLIN-5597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739826#comment-17739826
]
ASF GitHub Bot commented on KYLIN-5597:
---
Mrhs121 opened a new pull request, #2132:
URL: https://github.com/apache/kylin/pull/2132
## Proposed changes
Describe the big picture of your changes here to communicate to the
maintainers why we should accept this pull request. If it fixes a bug or
resolves a feature request, be sure to link to that issue.
## Branch to commit
- [ ] Branch **kylin3** for v2.x to v3.x
- [ ] Branch **kylin4** for v4.x
- [x] Branch **kylin5** for v5.x
## Types of changes
What types of changes does your code introduce to Kylin?
_Put an `x` in the boxes that apply_
- [x] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [x] I have created an issue on [Kylin's
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the
bug/feature there in detail
- [x] Commit messages in my PR start with the related jira ID, like
"KYLIN- Make Kylin project open-source"
- [x] Compiling and unit tests pass locally with my changes
- [x] I have added tests that prove my fix is effective or that my feature
works
- [x] I have added necessary documentation (if appropriate)
- [x] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the
solution you did and what alternatives you considered, etc...
> When the /etc/hosts file of the Kylin node does not configure the IP
> corresponding to the cluster domain name in the hadoop conf configuration
> file, Kylin will fail to start
> -
>
> Key: KYLIN-5597
> URL: https://issues.apache.org/jira/browse/KYLIN-5597
> Project: Kylin
> Issue Type: Bug
> Components: Environment , Security
>Affects Versions: 5.0-alpha
>Reporter: huangsheng
>Assignee: huangsheng
>Priority: Major
> Fix For: 5.0-alpha
>
>
> If the IP corresponding to the cluster domain name in the hadoop conf
> configuration file is not configured in the /etc/hosts file of the newly
> added KE node, the command
> {code:java}
> get-properties.sh kylin.kerberos.enabled{code}
> will fail when the service starts, and throw java.net.UnknownHostException
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5596) kylin5 dashboard test units error issue
[ https://issues.apache.org/jira/browse/KYLIN-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17739006#comment-17739006 ] ASF GitHub Bot commented on KYLIN-5596: --- hit-lacus merged PR #2129: URL: https://github.com/apache/kylin/pull/2129 > kylin5 dashboard test units error issue > --- > > Key: KYLIN-5596 > URL: https://issues.apache.org/jira/browse/KYLIN-5596 > Project: Kylin > Issue Type: Test >Affects Versions: 5.0-beta >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > > main branch code updated, but some code of dashboard is not been consistent > with the main branch, the test units need to fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5596) kylin5 dashboard test units error issue
[ https://issues.apache.org/jira/browse/KYLIN-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738994#comment-17738994 ] ASF GitHub Bot commented on KYLIN-5596: --- Syleechan opened a new pull request, #2129: URL: https://github.com/apache/kylin/pull/2129 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > kylin5 dashboard test units error issue > --- > > Key: KYLIN-5596 > URL: https://issues.apache.org/jira/browse/KYLIN-5596 > Project: Kylin > Issue Type: Test >Affects Versions: 5.0-beta >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > > main branch code updated, but some code of dashboard is not been consistent > with the main branch, the test units need to fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5465) kylin5 Embedded Dashboard for Query and Job
[ https://issues.apache.org/jira/browse/KYLIN-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738921#comment-17738921 ] ASF GitHub Bot commented on KYLIN-5465: --- hit-lacus commented on PR #2097: URL: https://github.com/apache/kylin/pull/2097#issuecomment-1614214443 Merged, thanks @Syleechan . > kylin5 Embedded Dashboard for Query and Job > --- > > Key: KYLIN-5465 > URL: https://issues.apache.org/jira/browse/KYLIN-5465 > Project: Kylin > Issue Type: New Feature > Components: Metrics >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: dashboard.png > > > 1.display job, cube statistics data. > 2.there have two query tables, if the query sql is error, then realization > will not store the data, so we use the realization table to display > statistics, because the error query will not include latency results. > 3.base on quote 2, we add some columns to realization table to collect data -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5465) kylin5 Embedded Dashboard for Query and Job
[ https://issues.apache.org/jira/browse/KYLIN-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738919#comment-17738919 ] ASF GitHub Bot commented on KYLIN-5465: --- hit-lacus merged PR #2097: URL: https://github.com/apache/kylin/pull/2097 > kylin5 Embedded Dashboard for Query and Job > --- > > Key: KYLIN-5465 > URL: https://issues.apache.org/jira/browse/KYLIN-5465 > Project: Kylin > Issue Type: New Feature > Components: Metrics >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: dashboard.png > > > 1.display job, cube statistics data. > 2.there have two query tables, if the query sql is error, then realization > will not store the data, so we use the realization table to display > statistics, because the error query will not include latency results. > 3.base on quote 2, we add some columns to realization table to collect data -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5595) [kylin 5.0] Launch Job Node not initialize spark session issue
[ https://issues.apache.org/jira/browse/KYLIN-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738900#comment-17738900 ] ASF GitHub Bot commented on KYLIN-5595: --- hit-lacus merged PR #2127: URL: https://github.com/apache/kylin/pull/2127 > [kylin 5.0] Launch Job Node not initialize spark session issue > -- > > Key: KYLIN-5595 > URL: https://issues.apache.org/jira/browse/KYLIN-5595 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: 5.0-beta >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: image (87).png, image (88).png > > > Saving model will execute 'checkFlatTableSql' method at job node, and it > will not skip the step by default. When execute 'checkFlatTableSql' method, > it will initialize spark session if the job node just started, the process of > getting spark session costs too much time. > The pic 87 shows that get spark session costs more than 63s, and the > execution of checking sql costs more than 2s, it is not friendly for saving > model first time after node launched, and it is also unreasonable. > So I suggest that the job node's process of spark session initialization > should be consistent with the query node, it means that the spark session > should be initialized as the node just started. And the spark session is a > singleton model, just need once initialization, as the pic 88 display. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5590) spark cube job supports priority, add job execution limit
[ https://issues.apache.org/jira/browse/KYLIN-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738542#comment-17738542 ] ASF GitHub Bot commented on KYLIN-5590: --- CodeCooker17 opened a new pull request, #2128: URL: https://github.com/apache/kylin/pull/2128 ## Proposed changes spark cube job supports priority, add job execution limit ## Branch to commit - [ ] Branch **kylin4** for v4.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > spark cube job supports priority, add job execution limit > - > > Key: KYLIN-5590 > URL: https://issues.apache.org/jira/browse/KYLIN-5590 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v4.0.1 >Reporter: Chuang Lee >Assignee: Chuang Lee >Priority: Major > > spark cube job supports priority, add job execution limit parallelism limit > and execution time period limit to prevent excessive cluster resources -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5595) [kylin 5.0] Launch Job Node not initialize spark session issue
[ https://issues.apache.org/jira/browse/KYLIN-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738009#comment-17738009 ] ASF GitHub Bot commented on KYLIN-5595: --- Syleechan opened a new pull request, #2127: URL: https://github.com/apache/kylin/pull/2127 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > [kylin 5.0] Launch Job Node not initialize spark session issue > -- > > Key: KYLIN-5595 > URL: https://issues.apache.org/jira/browse/KYLIN-5595 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: 5.0-beta >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: image (87).png, image (88).png > > > Saving model will execute 'checkFlatTableSql' method at job node, and it > will not skip the step by default. When execute 'checkFlatTableSql' method, > it will initialize spark session if the job node just started, the process of > getting spark session costs too much time. > The pic 87 shows that get spark session costs more than 63s, and the > execution of checking sql costs more than 2s, it is not friendly for saving > model first time after node launched, and it is also unreasonable. > So I suggest that the job node's process of initialization spark session > should be consistent with the query node, it means that the spark session > should be initialized as the node just started. And the spark session is a > singleton model, just need once initialization, as the pic 88 display. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5514) Support batch build Cube and batch refresh Cube
[ https://issues.apache.org/jira/browse/KYLIN-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737128#comment-17737128 ] ASF GitHub Bot commented on KYLIN-5514: --- CodeCooker17 opened a new pull request, #2126: URL: https://github.com/apache/kylin/pull/2126 …ch controls to part overlaps refresh. [KYLIN-5514][feature] support build or refresh type, support batch build by nature month. ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Support batch build Cube and batch refresh Cube > --- > > Key: KYLIN-5514 > URL: https://issues.apache.org/jira/browse/KYLIN-5514 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, REST Service >Affects Versions: v4.0.1 >Reporter: Chuang Lee >Assignee: Chuang Lee >Priority: Major > > At present, there is no function to re-refresh Cube segments in batches > within a large time range. If re-refresh needs to select segments for > re-refresh, it will be very energy-consuming. At the same time, it does not > support large time range Cube for segment construction and splitting > according to small range. Therefore, it is hoped to support the following > functions: (1) The user can refresh all segments within the corresponding > time range of the Cube by selecting a large time range. (2) When the user > selects a large time range to build a segment, the segment can be built step > by step according to the granularity of the merge. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5527) Kylin job engine adds the ability to check entries with data source(e.g. hive)
[ https://issues.apache.org/jira/browse/KYLIN-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731958#comment-17731958 ] ASF GitHub Bot commented on KYLIN-5527: --- hit-lacus merged PR #2124: URL: https://github.com/apache/kylin/pull/2124 > Kylin job engine adds the ability to check entries with data source(e.g. hive) > -- > > Key: KYLIN-5527 > URL: https://issues.apache.org/jira/browse/KYLIN-5527 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Critical > Fix For: 5.0-beta > > Attachments: (Chinese)KYLIN-5527 Kylin job engine adds the ability to > check entries with data source(e.g. hive).pdf, (English) KYLIN-5527 Kylin job > engine adds the ability to check entries with data source(e.g. hive).pdf > > > After adding some new indexes in the Kylin5 model, the operation of > completing the indexes is needed. If the previous data in the data source > (e.g.:hive) has been deleted, then there will be a situation that the newly > added index data is empty, which will lead to wrong query results. Here we > need to add a new data source check function, if the data in the data source > has been deleted, then do not fill in the data of the corresponding time > segment to ensure the correctness of the query results. > > 在Kylin5的模型中新增加一些索引后，需要进行补齐索引的操作，如果数据源（如:hive）里之前的数据被删掉了，那么就会出现新补出来的索引数据为空的情况，从而导致查询结果有误。这里需要新增加一个数据源核对的功能，如果数据源中的数据已经被删掉了，那么就不去补对应时间segment的数据，保证查询结果的正确性。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5527) Kylin job engine adds the ability to check entries with data source(e.g. hive)
[ https://issues.apache.org/jira/browse/KYLIN-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731957#comment-17731957 ] ASF GitHub Bot commented on KYLIN-5527: --- hit-lacus commented on PR #2124: URL: https://github.com/apache/kylin/pull/2124#issuecomment-1588808866 LGTM > Kylin job engine adds the ability to check entries with data source(e.g. hive) > -- > > Key: KYLIN-5527 > URL: https://issues.apache.org/jira/browse/KYLIN-5527 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Critical > Fix For: 5.0-beta > > Attachments: (Chinese)KYLIN-5527 Kylin job engine adds the ability to > check entries with data source(e.g. hive).pdf, (English) KYLIN-5527 Kylin job > engine adds the ability to check entries with data source(e.g. hive).pdf > > > After adding some new indexes in the Kylin5 model, the operation of > completing the indexes is needed. If the previous data in the data source > (e.g.:hive) has been deleted, then there will be a situation that the newly > added index data is empty, which will lead to wrong query results. Here we > need to add a new data source check function, if the data in the data source > has been deleted, then do not fill in the data of the corresponding time > segment to ensure the correctness of the query results. > > 在Kylin5的模型中新增加一些索引后，需要进行补齐索引的操作，如果数据源（如:hive）里之前的数据被删掉了，那么就会出现新补出来的索引数据为空的情况，从而导致查询结果有误。这里需要新增加一个数据源核对的功能，如果数据源中的数据已经被删掉了，那么就不去补对应时间segment的数据，保证查询结果的正确性。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5527) Kylin job engine adds the ability to check entries with data source(e.g. hive)
[ https://issues.apache.org/jira/browse/KYLIN-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730800#comment-17730800 ] ASF GitHub Bot commented on KYLIN-5527: --- fengguangyuan commented on PR #2124: URL: https://github.com/apache/kylin/pull/2124#issuecomment-1583908138 retest this please > Kylin job engine adds the ability to check entries with data source(e.g. hive) > -- > > Key: KYLIN-5527 > URL: https://issues.apache.org/jira/browse/KYLIN-5527 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Critical > Fix For: 5.0-beta > > Attachments: (Chinese)KYLIN-5527 Kylin job engine adds the ability to > check entries with data source(e.g. hive).pdf, (English) KYLIN-5527 Kylin job > engine adds the ability to check entries with data source(e.g. hive).pdf > > > After adding some new indexes in the Kylin5 model, the operation of > completing the indexes is needed. If the previous data in the data source > (e.g.:hive) has been deleted, then there will be a situation that the newly > added index data is empty, which will lead to wrong query results. Here we > need to add a new data source check function, if the data in the data source > has been deleted, then do not fill in the data of the corresponding time > segment to ensure the correctness of the query results. > > 在Kylin5的模型中新增加一些索引后，需要进行补齐索引的操作，如果数据源（如:hive）里之前的数据被删掉了，那么就会出现新补出来的索引数据为空的情况，从而导致查询结果有误。这里需要新增加一个数据源核对的功能，如果数据源中的数据已经被删掉了，那么就不去补对应时间segment的数据，保证查询结果的正确性。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[
https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730799#comment-17730799
]
ASF GitHub Bot commented on KYLIN-5545:
---
fengguangyuan closed pull request #2122: KYLIN-5545 Try best to interrupt
running queries and limit the number of queries to run.
URL: https://github.com/apache/kylin/pull/2122
> Try best to interrupt running queries and limit the number of queries to run
>
>
> Key: KYLIN-5545
> URL: https://issues.apache.org/jira/browse/KYLIN-5545
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: fanfanAlice
>Priority: Major
> Fix For: 5.0-alpha
>
>
> *Issues*
> There are two issues will decrease the stability of Kylin:
> # Missed some places to cancel the planning/running queries if occurred
> interruptions, leading to the useless consumptions of calculating resources.
> # Don't limit the number of queries that can be pushed down, leading to too
> many queries to run, evenly to a OOM.
> So it's great to have a PR to improve/solve the above issues.
>
> *Proposal changes*
> # Add a new property
> {{kylin.query.pushdown-concurrent-running-threshold=10}} to control the
> concurrency of the queries pushed down, 10 running queries by default, which
> will be bond to a Semaphore instance in JVM scope.
> # The pushed down query will calling {{Semaphore}} instance's
> {{tryAcquire(long timeout, TimeUnit unit)}} to acquire the chance to run, so
> the waiting queries could be interrupted if failed in {{330s}} by default.
>
> *Why changes*
> # {{QueryLimiter}} won't working then the server is not in {{downgrade}}
> state, which means no limitations on the number of queries pushed down to
> run, therefore we need another strategy to accomplish the goal.
> # During retrieving the file status, it gives us a chance to react to the
> interruption, so as to terminate the planning queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[
https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730798#comment-17730798
]
ASF GitHub Bot commented on KYLIN-5545:
---
fengguangyuan commented on PR #2122:
URL: https://github.com/apache/kylin/pull/2122#issuecomment-1583904643
Close this PR because it has been attached into
https://github.com/apache/kylin/pull/2124
> Try best to interrupt running queries and limit the number of queries to run
>
>
> Key: KYLIN-5545
> URL: https://issues.apache.org/jira/browse/KYLIN-5545
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: fanfanAlice
>Priority: Major
> Fix For: 5.0-alpha
>
>
> *Issues*
> There are two issues will decrease the stability of Kylin:
> # Missed some places to cancel the planning/running queries if occurred
> interruptions, leading to the useless consumptions of calculating resources.
> # Don't limit the number of queries that can be pushed down, leading to too
> many queries to run, evenly to a OOM.
> So it's great to have a PR to improve/solve the above issues.
>
> *Proposal changes*
> # Add a new property
> {{kylin.query.pushdown-concurrent-running-threshold=10}} to control the
> concurrency of the queries pushed down, 10 running queries by default, which
> will be bond to a Semaphore instance in JVM scope.
> # The pushed down query will calling {{Semaphore}} instance's
> {{tryAcquire(long timeout, TimeUnit unit)}} to acquire the chance to run, so
> the waiting queries could be interrupted if failed in {{330s}} by default.
>
> *Why changes*
> # {{QueryLimiter}} won't working then the server is not in {{downgrade}}
> state, which means no limitations on the number of queries pushed down to
> run, therefore we need another strategy to accomplish the goal.
> # During retrieving the file status, it gives us a chance to react to the
> interruption, so as to terminate the planning queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5546) Support to cancel asynchronous query job based on the asynchronous query id
[
https://issues.apache.org/jira/browse/KYLIN-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730797#comment-17730797
]
ASF GitHub Bot commented on KYLIN-5546:
---
fengguangyuan closed pull request #2121: KYLIN-5546 Add stop async query job
api.
URL: https://github.com/apache/kylin/pull/2121
> Support to cancel asynchronous query job based on the asynchronous query id
> ---
>
> Key: KYLIN-5546
> URL: https://issues.apache.org/jira/browse/KYLIN-5546
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: Guangyuan Feng
>Priority: Major
> Fix For: 5.0-alpha
>
>
> h2. Issues
> Currently, Kylin has no open APIs to cancel/kill the asynchronous running
> query, users have to dig into the log and find out the Spark process, then to
> kill the process manually.
> Without the entrypoint to kill the running asynchronous queries, the server
> will lose control of resources, finally enter into a unstable state.
> So I think it's great for Kylin to supply such an open API, to cancel/kill
> the asynchronous submitted queries.
> h2. Proposal changes
> Supply a *DELETE* REST API {{{}/async_query/stop/{query_id:.+{ to kill
> the submitted asynchronous query by the {{{}query id{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5546) Support to cancel asynchronous query job based on the asynchronous query id
[
https://issues.apache.org/jira/browse/KYLIN-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730796#comment-17730796
]
ASF GitHub Bot commented on KYLIN-5546:
---
fengguangyuan commented on PR #2121:
URL: https://github.com/apache/kylin/pull/2121#issuecomment-1583904419
Close this PR because it has been attached in
https://github.com/apache/kylin/pull/2124
> Support to cancel asynchronous query job based on the asynchronous query id
> ---
>
> Key: KYLIN-5546
> URL: https://issues.apache.org/jira/browse/KYLIN-5546
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: Guangyuan Feng
>Priority: Major
> Fix For: 5.0-alpha
>
>
> h2. Issues
> Currently, Kylin has no open APIs to cancel/kill the asynchronous running
> query, users have to dig into the log and find out the Spark process, then to
> kill the process manually.
> Without the entrypoint to kill the running asynchronous queries, the server
> will lose control of resources, finally enter into a unstable state.
> So I think it's great for Kylin to supply such an open API, to cancel/kill
> the asynchronous submitted queries.
> h2. Proposal changes
> Supply a *DELETE* REST API {{{}/async_query/stop/{query_id:.+{ to kill
> the submitted asynchronous query by the {{{}query id{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5527) Kylin job engine adds the ability to check entries with data source(e.g. hive)
[ https://issues.apache.org/jira/browse/KYLIN-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730795#comment-17730795 ] ASF GitHub Bot commented on KYLIN-5527: --- fengguangyuan opened a new pull request, #2124: URL: https://github.com/apache/kylin/pull/2124 ## Proposed changes https://issues.apache.org/jira/browse/KYLIN-5588 https://issues.apache.org/jira/browse/KYLIN-5587 https://issues.apache.org/jira/browse/KYLIN-5586 https://issues.apache.org/jira/browse/KYLIN-5585 https://issues.apache.org/jira/browse/KYLIN-5584 https://issues.apache.org/jira/browse/KYLIN-5583 https://issues.apache.org/jira/browse/KYLIN-5582 https://issues.apache.org/jira/browse/KYLIN-5581 https://issues.apache.org/jira/browse/KYLIN-5580 https://issues.apache.org/jira/browse/KYLIN-5579 https://issues.apache.org/jira/browse/KYLIN-5578 https://issues.apache.org/jira/browse/KYLIN-5577 https://issues.apache.org/jira/browse/KYLIN-5576 https://issues.apache.org/jira/browse/KYLIN-5575 https://issues.apache.org/jira/browse/KYLIN-5574 https://issues.apache.org/jira/browse/KYLIN-5573 https://issues.apache.org/jira/browse/KYLIN-5572 https://issues.apache.org/jira/browse/KYLIN-5571 https://issues.apache.org/jira/browse/KYLIN-5570 https://issues.apache.org/jira/browse/KYLIN-5569 https://issues.apache.org/jira/browse/KYLIN-5568 https://issues.apache.org/jira/browse/KYLIN-5567 https://issues.apache.org/jira/browse/KYLIN-5566 https://issues.apache.org/jira/browse/KYLIN-5565 https://issues.apache.org/jira/browse/KYLIN-5564 https://issues.apache.org/jira/browse/KYLIN-5563 https://issues.apache.org/jira/browse/KYLIN-5562 https://issues.apache.org/jira/browse/KYLIN-5561 https://issues.apache.org/jira/browse/KYLIN-5560 https://issues.apache.org/jira/browse/KYLIN-5559 https://issues.apache.org/jira/browse/KYLIN-5558 https://issues.apache.org/jira/browse/KYLIN-5557 https://issues.apache.org/jira/browse/KYLIN-5556 https://issues.apache.org/jira/browse/KYLIN- https://issues.apache.org/jira/browse/KYLIN-5546 https://issues.apache.org/jira/browse/KYLIN-5545 https://issues.apache.org/jira/browse/KYLIN-5527 ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [x] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Kylin job engine adds the ability to check entries with data source(e.g. hive) > -- > > Key: KYLIN-5527 > URL: https://issues.apache.org/jira/browse/KYLIN-5527 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Critical > Fix For: 5.0-beta > > Attachments: (Chinese)KYLIN-5527 Kylin job engine adds the ability to > check entries with data source(e.g. hive).pdf, (English) KYLIN-5527 Kylin job > engine adds the ability to check entries with data source(e.g. hive).pdf > > > After adding some new indexes in the Kylin5 model, the operation of > completing the indexes is needed. If the previous data in the data source > (e.g.:hive) has been deleted, then there will be a situation that the newly > added index data is empty, which will lead to wrong
[jira] [Commented] (KYLIN-5541) Provide a convenient operation to deal with the situation where many dimensions (hundreds) need to be selected in the "Edit Include Dimensions" of the aggregation group
[ https://issues.apache.org/jira/browse/KYLIN-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729944#comment-17729944 ] ASF GitHub Bot commented on KYLIN-5541: --- hit-lacus merged PR #2123: URL: https://github.com/apache/kylin/pull/2123 > Provide a convenient operation to deal with the situation where many > dimensions (hundreds) need to be selected in the "Edit Include Dimensions" of > the aggregation group > > > Key: KYLIN-5541 > URL: https://issues.apache.org/jira/browse/KYLIN-5541 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-06-02-14-42-03-922.png, > image-2023-06-02-15-35-30-466.png, image-2023-06-02-15-35-53-373.png, > image-2023-06-02-15-36-11-332.png, image-2023-06-02-15-36-27-702.png, > image-2023-06-02-15-36-44-045.png, image-2023-06-02-15-36-57-296.png > > > !image-2023-06-02-15-35-30-466.png! > !image-2023-06-02-15-35-53-373.png! > !image-2023-06-02-15-36-11-332.png! > !image-2023-06-02-15-36-27-702.png! > !image-2023-06-02-15-36-57-296.png! > !image-2023-06-02-15-36-44-045.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5541) Provide a convenient operation to deal with the situation where many dimensions (hundreds) need to be selected in the "Edit Include Dimensions" of the aggregation group
[ https://issues.apache.org/jira/browse/KYLIN-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729674#comment-17729674 ] ASF GitHub Bot commented on KYLIN-5541: --- LauraXia123 opened a new pull request, #2123: URL: https://github.com/apache/kylin/pull/2123 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Provide a convenient operation to deal with the situation where many > dimensions (hundreds) need to be selected in the "Edit Include Dimensions" of > the aggregation group > > > Key: KYLIN-5541 > URL: https://issues.apache.org/jira/browse/KYLIN-5541 > Project: Kylin > Issue Type: Bug >Reporter: Laura Xia >Assignee: Laura Xia >Priority: Major > Fix For: 5.0-beta > > Attachments: image-2023-06-02-14-42-03-922.png, > image-2023-06-02-15-35-30-466.png, image-2023-06-02-15-35-53-373.png, > image-2023-06-02-15-36-11-332.png, image-2023-06-02-15-36-27-702.png, > image-2023-06-02-15-36-44-045.png, image-2023-06-02-15-36-57-296.png > > > !image-2023-06-02-15-35-30-466.png! > !image-2023-06-02-15-35-53-373.png! > !image-2023-06-02-15-36-11-332.png! > !image-2023-06-02-15-36-27-702.png! > !image-2023-06-02-15-36-57-296.png! > !image-2023-06-02-15-36-44-045.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[
https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729650#comment-17729650
]
ASF GitHub Bot commented on KYLIN-5545:
---
hit-lacus commented on PR #2122:
URL: https://github.com/apache/kylin/pull/2122#issuecomment-1578148858
retest this please
> Try best to interrupt running queries and limit the number of queries to run
>
>
> Key: KYLIN-5545
> URL: https://issues.apache.org/jira/browse/KYLIN-5545
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: fanfanAlice
>Priority: Major
> Fix For: 5.0-alpha
>
>
> *Issues*
> There are two issues will decrease the stability of Kylin:
> # Missed some places to cancel the planning/running queries if occurred
> interruptions, leading to the useless consumptions of calculating resources.
> # Don't limit the number of queries that can be pushed down, leading to too
> many queries to run, evenly to a OOM.
> So it's great to have a PR to improve/solve the above issues.
>
> *Proposal changes*
> # Add a new property
> {{kylin.query.pushdown-concurrent-running-threshold=10}} to control the
> concurrency of the queries pushed down, 10 running queries by default, which
> will be bond to a Semaphore instance in JVM scope.
> # The pushed down query will calling {{Semaphore}} instance's
> {{tryAcquire(long timeout, TimeUnit unit)}} to acquire the chance to run, so
> the waiting queries could be interrupted if failed in {{330s}} by default.
>
> *Why changes*
> # {{QueryLimiter}} won't working then the server is not in {{downgrade}}
> state, which means no limitations on the number of queries pushed down to
> run, therefore we need another strategy to accomplish the goal.
> # During retrieving the file status, it gives us a chance to react to the
> interruption, so as to terminate the planning queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5465) kylin5 Embedded Dashboard for Query and Job
[ https://issues.apache.org/jira/browse/KYLIN-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729649#comment-17729649 ] ASF GitHub Bot commented on KYLIN-5465: --- hit-lacus commented on PR #2097: URL: https://github.com/apache/kylin/pull/2097#issuecomment-1578148260 retest this please > kylin5 Embedded Dashboard for Query and Job > --- > > Key: KYLIN-5465 > URL: https://issues.apache.org/jira/browse/KYLIN-5465 > Project: Kylin > Issue Type: New Feature > Components: Metrics >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: dashboard.png > > > 1.display job, cube statistics data. > 2.there have two query tables, if the query sql is error, then realization > will not store the data, so we use the realization table to display > statistics, because the error query will not include latency results. > 3.base on quote 2, we add some columns to realization table to collect data -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5465) kylin5 Embedded Dashboard for Query and Job
[ https://issues.apache.org/jira/browse/KYLIN-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729648#comment-17729648 ] ASF GitHub Bot commented on KYLIN-5465: --- hit-lacus commented on PR #2097: URL: https://github.com/apache/kylin/pull/2097#issuecomment-1578137808 1234 > kylin5 Embedded Dashboard for Query and Job > --- > > Key: KYLIN-5465 > URL: https://issues.apache.org/jira/browse/KYLIN-5465 > Project: Kylin > Issue Type: New Feature > Components: Metrics >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: dashboard.png > > > 1.display job, cube statistics data. > 2.there have two query tables, if the query sql is error, then realization > will not store the data, so we use the realization table to display > statistics, because the error query will not include latency results. > 3.base on quote 2, we add some columns to realization table to collect data -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5465) kylin5 Embedded Dashboard for Query and Job
[ https://issues.apache.org/jira/browse/KYLIN-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729642#comment-17729642 ] ASF GitHub Bot commented on KYLIN-5465: --- hit-lacus commented on PR #2097: URL: https://github.com/apache/kylin/pull/2097#issuecomment-1578123372 retest this please > kylin5 Embedded Dashboard for Query and Job > --- > > Key: KYLIN-5465 > URL: https://issues.apache.org/jira/browse/KYLIN-5465 > Project: Kylin > Issue Type: New Feature > Components: Metrics >Reporter: Li Can >Assignee: Li Can >Priority: Major > Fix For: 5.0-beta > > Attachments: dashboard.png > > > 1.display job, cube statistics data. > 2.there have two query tables, if the query sql is error, then realization > will not store the data, so we use the realization table to display > statistics, because the error query will not include latency results. > 3.base on quote 2, we add some columns to realization table to collect data -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[ https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729620#comment-17729620 ] ASF GitHub Bot commented on KYLIN-5545: --- hit-lacus commented on PR #2122: URL: https://github.com/apache/kylin/pull/2122#issuecomment-1578046601 retest this please > Try best to interrupt running queries and limit the number of queries to run > > > Key: KYLIN-5545 > URL: https://issues.apache.org/jira/browse/KYLIN-5545 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: 5.0-alpha >Reporter: Guangyuan Feng >Assignee: fanfanAlice >Priority: Major > Fix For: 5.0-alpha > > > There are two issues will decrease the stability of Kylin: > # Missed some places to cancel the planning/running queries if occurred > interruptions, leading to the useless consumptions of calculating resources. > # Don't limit the number of queries that can be pushed down, leading to too > many queries to run, evenly to a OOM. > So it's great to have a PR to improve/solve the above issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[ https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729615#comment-17729615 ] ASF GitHub Bot commented on KYLIN-5545: --- hit-lacus commented on PR #2122: URL: https://github.com/apache/kylin/pull/2122#issuecomment-1578030775 retest this please > Try best to interrupt running queries and limit the number of queries to run > > > Key: KYLIN-5545 > URL: https://issues.apache.org/jira/browse/KYLIN-5545 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: 5.0-alpha >Reporter: Guangyuan Feng >Assignee: fanfanAlice >Priority: Major > Fix For: 5.0-alpha > > > There are two issues will decrease the stability of Kylin: > # Missed some places to cancel the planning/running queries if occurred > interruptions, leading to the useless consumptions of calculating resources. > # Don't limit the number of queries that can be pushed down, leading to too > many queries to run, evenly to a OOM. > So it's great to have a PR to improve/solve the above issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5545) Try best to interrupt running queries and limit the number of queries to run
[ https://issues.apache.org/jira/browse/KYLIN-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729605#comment-17729605 ] ASF GitHub Bot commented on KYLIN-5545: --- fengguangyuan opened a new pull request, #2122: URL: https://github.com/apache/kylin/pull/2122 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Branch to commit - [ ] Branch **kylin3** for v2.x to v3.x - [ ] Branch **kylin4** for v4.x - [ ] Branch **kylin5** for v5.x ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Try best to interrupt running queries and limit the number of queries to run > > > Key: KYLIN-5545 > URL: https://issues.apache.org/jira/browse/KYLIN-5545 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: 5.0-alpha >Reporter: Guangyuan Feng >Priority: Major > Fix For: 5.0-alpha > > > There are two issues will decrease the stability of Kylin: > # Don't try best to cancel the running queries if occurred interruptions, > leading to the useless consumptions of calculating resources. > # Don't limit the number of queries that can be pushed down, leading to too > many queries to run, evenly to a OOM. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5546) Support to cancel asynchronous query job based on the asynchronous query id
[
https://issues.apache.org/jira/browse/KYLIN-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729598#comment-17729598
]
ASF GitHub Bot commented on KYLIN-5546:
---
fengguangyuan opened a new pull request, #2121:
URL: https://github.com/apache/kylin/pull/2121
## Proposed changes
As described in JIRA
[KYLIN-5546](https://issues.apache.org/jira/browse/KYLIN-5546)，this PR supply a
DELETE REST API `/async_query/stop/{query_id:.+}` to kill the submitted
asynchronous query by the `query id`.
## Branch to commit
- [ ] Branch **kylin3** for v2.x to v3.x
- [ ] Branch **kylin4** for v4.x
- [ ] Branch **kylin5** for v5.x
## Types of changes
What types of changes does your code introduce to Kylin?
_Put an `x` in the boxes that apply_
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation Update (if none of the other choices apply)
## Checklist
_Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look for
before merging your code._
- [ ] I have created an issue on [Kylin's
jira](https://issues.apache.org/jira/browse/KYLIN), and have described the
bug/feature there in detail
- [ ] Commit messages in my PR start with the related jira ID, like
"KYLIN- Make Kylin project open-source"
- [ ] Compiling and unit tests pass locally with my changes
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] I have added necessary documentation (if appropriate)
- [ ] Any dependent changes have been merged
## Further comments
If this is a relatively large or complex change, kick off the discussion at
u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the
solution you did and what alternatives you considered, etc...
> Support to cancel asynchronous query job based on the asynchronous query id
> ---
>
> Key: KYLIN-5546
> URL: https://issues.apache.org/jira/browse/KYLIN-5546
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: 5.0-alpha
>Reporter: Guangyuan Feng
>Assignee: Guangyuan Feng
>Priority: Major
> Fix For: 5.0-alpha
>
>
> Currently, Kylin has no open APIs to cancel/kill the asynchronous running
> query, users have to dig into the log and find out the Spark process, then to
> kill the process manually.
> Without the entrypoint to kill the running asynchronous queries, the server
> will lose control of resources, finally enter into a unstable state.
> So I think it's great for Kylin to supply such an open API, to cancel/kill
> the asynchronous submitted queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5521) JoinsGraph optimization: Query SQL association order change causes the model to fail to hit
[ https://issues.apache.org/jira/browse/KYLIN-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720120#comment-17720120 ] ASF GitHub Bot commented on KYLIN-5521: --- hit-lacus merged PR #2116: URL: https://github.com/apache/kylin/pull/2116 > JoinsGraph optimization: Query SQL association order change causes the model > to fail to hit > --- > > Key: KYLIN-5521 > URL: https://issues.apache.org/jira/browse/KYLIN-5521 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: 5.0-alpha >Reporter: Yaguang Jia >Assignee: Yaguang Jia >Priority: Major > Fix For: 5.0-beta > > Attachments: (Chinese)KYLIN-5521 JoinsGraph optimization_ Query SQL > association order change causes the model to fail to hit.pdf, > (English)KYLIN-5521 JoinsGraph optimization_ Query SQL association order > change causes the model to fail to hit.pdf > > > In the log, the following two query statements, only the order of the > association of the first 2 tables is changed (the actual logical relationship > has not changed), resulting in the query statement can not be hit by the same > index. When there are only 2 tables after from, the order of the replacement > table association can hit the same index. > > 日志中，以下两条查询语句，只是前面2张表的关联先后顺序换了一下（实际逻辑关系并未改变），导致查询语句不能被同一个索引击中。而当from后面只有2张表时，替换表关联的先后顺序是可以击中同一个索引的。 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
