[jira] [Updated] (KYLIN-3074) Change cube access to project access in ExternalAclProvider.java
[ https://issues.apache.org/jira/browse/KYLIN-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peng.jianhua updated KYLIN-3074: Attachment: 0001-KYLIN-3074-branch-2.2.x.patch > Change cube access to project access in ExternalAclProvider.java > > > Key: KYLIN-3074 > URL: https://issues.apache.org/jira/browse/KYLIN-3074 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.2.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Attachments: 0001-KYLIN-3074-branch-2.2.x.patch > > > According to ranger recommendations, the following aspects were modified: > 1.Cube ACL is removed since v2.2.0, we shuold modify the aclPermission name, > no longer cube access. > 2.Remove unnecessary requests. For example, get cube and model access > entities, it will be recorded in the ranger audit log. > 3.Remove unnecessary permission verification. Kylin currently used four > permissions(ADMINISTRATION,MANAGEMENT,OPERATION,READ), other permissions do > not need to be verified. Every time permission verification will be recorded > in the ranger audit log. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3074) Change cube access to project access in ExternalAclProvider.java
[ https://issues.apache.org/jira/browse/KYLIN-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peng.jianhua updated KYLIN-3074: Affects Version/s: v2.2.0 Priority: Major (was: Minor) Description: According to ranger recommendations, the following aspects were modified: 1.Cube ACL is removed since v2.2.0, we shuold modify the aclPermission name, no longer cube access. 2.Remove unnecessary requests. For example, get cube and model access entities, it will be recorded in the ranger audit log. 3.Remove unnecessary permission verification. Kylin currently used four permissions(ADMINISTRATION,MANAGEMENT,OPERATION,READ), other permissions do not need to be verified. Every time permission verification will be recorded in the ranger audit log. > Change cube access to project access in ExternalAclProvider.java > > > Key: KYLIN-3074 > URL: https://issues.apache.org/jira/browse/KYLIN-3074 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.2.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Attachments: 0001-KYLIN-3074-branch-2.2.x.patch > > > According to ranger recommendations, the following aspects were modified: > 1.Cube ACL is removed since v2.2.0, we shuold modify the aclPermission name, > no longer cube access. > 2.Remove unnecessary requests. For example, get cube and model access > entities, it will be recorded in the ranger audit log. > 3.Remove unnecessary permission verification. Kylin currently used four > permissions(ADMINISTRATION,MANAGEMENT,OPERATION,READ), other permissions do > not need to be verified. Every time permission verification will be recorded > in the ranger audit log. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3061) When we cancel the Topic modification for 'Kafka Setting' of streaming table, the 'Cancel' operation will make a mistake.
[
https://issues.apache.org/jira/browse/KYLIN-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276347#comment-16276347
]
peng.jianhua commented on KYLIN-3061:
-
Hi [~Zhixiong Chen], I'm sorry that I haven't understood your suggestion.
'add cluster' means that you should add a new record, but 'edit cluster' means
that you should modified a existent record, so how to make them in same logic?
so could you explain it in more detail? Thanks!
> When we cancel the Topic modification for 'Kafka Setting' of streaming table,
> the 'Cancel' operation will make a mistake.
> -
>
> Key: KYLIN-3061
> URL: https://issues.apache.org/jira/browse/KYLIN-3061
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.3.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Attachments:
> 0001-KYLIN-3061-When-we-cancel-the-Topic-modification-for.patch,
> can_add_the_same_record.png, cancel_is_not_right.png, edit_kafka_setting.png,
> edit_streaming_table.png
>
>
> There are two bugs in this issue.
> First
> 1. Choose one streaming table, choose the 'Streaming Cluster' tab, then click
> 'Edit' button, refer to [^edit_streaming_table.png]
> 2. The edit page will be opened, then you can edit the value of topic of
> 'Kafka Setting'. As long as you modify the ID, Host, Port value, the original
> value tag will follow changes, refer to [^edit_kafka_setting.png];
> 3. When you click 'cancel' button, you will find the old values have been
> changed to the new values, and if you click the 'submit' button, you will
> also find the values to be canceled will be submitted, refer to
> [^cancel_is_not_right.png]
> But I think the correct way should be that 'cancel' button will not change
> any value.
> Second
> The follow code in streamingConfig.js has a bug, even if "cluster.newBroker"
> object matchs the one element of "cluster.brokers" array, this "if" decision
> will return false. Because even though the "cluster.newBroker" object has the
> same attribute values as an element in the array, their storage addresses may
> be different. The result is that you can add several same record, like
> [^can_add_the_same_record.png].
> {code:java}
> $scope.saveNewBroker = function(cluster) {
> if (cluster.brokers.indexOf(cluster.newBroker) === -1) {
> ..
> }
> }
> {code}
> So I have repaired these two bugs, please check the patch, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-3055) NullPointerException in MutableRoaringBitmap.or
[
https://issues.apache.org/jira/browse/KYLIN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen reassigned KYLIN-3055:
-
Assignee: yuchuqian (was: kangkaisen)
> NullPointerException in MutableRoaringBitmap.or
> ---
>
> Key: KYLIN-3055
> URL: https://issues.apache.org/jira/browse/KYLIN-3055
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.2.0
>Reporter: yuchuqian
>Assignee: yuchuqian
> Fix For: v2.3.0
>
> Attachments: KYLIN-3055.patch
>
>
> 2017-11-21 19:55:17,363 ERROR [Query
> b1fbcd45-6524-4b1e-8844-1d6d6277a1bf-120] service.QueryService:459 :
> Exception while executing query
> java.sql.SQLException: Error while executing SQL "select part_dt,
> intersect_count(item_count, part_dt, array[date'2012-01-01']) as first_day,
> intersect_count(item_count, part_dt, array[date'2012-01-02']) as second_day,
> intersect_count(item_count, part_dt, array[date'2012-01-03']) as third_day,
> intersect_count(item_count, part_dt,
> array[date'2012-01-01',date'2012-01-02']) as retention_oneday,
> intersect_count(item_count, part_dt,
> array[date'2012-01-01',date'2012-01-02',date'2012-01-03']) as retention_twoday
> from kylin_sales
> where part_dt in (date'2012-01-01',date'2012-01-02',date'2012-01-03')
> group by PART_DT
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:834)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:561)
> at org.apache.kylin.rest.service.QueryService.query(QueryService.java:181)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:415)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:78)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> ..
> Caused by: java.lang.NullPointerException
> at
> org.roaringbitmap.buffer.MutableRoaringBitmap.or(MutableRoaringBitmap.java:1041)
> at
> org.apache.kylin.measure.bitmap.RoaringBitmapCounter.orWith(RoaringBitmapCounter.java:72)
> at
> org.apache.kylin.measure.bitmap.BitmapIntersectDistinctCountAggFunc$RetentionPartialResult.add(BitmapIntersectDistinctCountAggFunc.java:57)
> at
> org.apache.kylin.measure.bitmap.BitmapIntersectDistinctCountAggFunc.add(BitmapIntersectDistinctCountAggFunc.java:90)
> at Baz$4.apply(ANONYMOUS.java:136)
> at Baz$4.apply(ANONYMOUS.java:158)
> at Baz$4.apply(ANONYMOUS.java)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:832)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
> at Baz.bind(Baz.java:99)
> How to re-produce:
> 1. run $KYLIN_HOME/bin/sample.sh
> 2. then create a cube like
> {
> "uuid": "9554f6f6-74dc-489e-b780-2f48f281576c",
> "last_modified": 1511247707372,
> "version": "2.2.0.0",
> "name": "test",
> "is_draft": false,
> "model_name": "kylin_sales_model",
> "description": "",
> "null_string": null,
> "dimensions": [
> {
> "name": "PART_DT",
> "table": "KYLIN_SALES",
> "column": "PART_DT",
> "derived": null
> },
> {
> "name": "LEAF_CATEG_ID",
> "table": "KYLIN_SALES",
> "column": "LEAF_CATEG_ID",
> "derived": null
> },
> {
> "name": "LSTG_SITE_ID",
> "table": "KYLIN_SALES",
> "column": "LSTG_SITE_ID",
> "derived": null
> },
> {
> "name": "CAL_DT",
> "table": "KYLIN_CAL_DT",
> "column": null,
> "derived": [
> "CAL_DT"
> ]
> },
> {
> "name": "LEAF_CATEG_ID",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
> "LEAF_CATEG_ID"
> ]
> },
> {
> "name": "USER_DEFINED_FIELD1",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
> "USER_DEFINED_FIELD1"
> ]
> },
> {
> "name": "USER_DEFINED_FIELD3",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
>
[jira] [Assigned] (KYLIN-3055) NullPointerException in MutableRoaringBitmap.or
[
https://issues.apache.org/jira/browse/KYLIN-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kangkaisen reassigned KYLIN-3055:
-
Assignee: kangkaisen (was: yuchuqian)
> NullPointerException in MutableRoaringBitmap.or
> ---
>
> Key: KYLIN-3055
> URL: https://issues.apache.org/jira/browse/KYLIN-3055
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.2.0
>Reporter: yuchuqian
>Assignee: kangkaisen
> Fix For: v2.3.0
>
> Attachments: KYLIN-3055.patch
>
>
> 2017-11-21 19:55:17,363 ERROR [Query
> b1fbcd45-6524-4b1e-8844-1d6d6277a1bf-120] service.QueryService:459 :
> Exception while executing query
> java.sql.SQLException: Error while executing SQL "select part_dt,
> intersect_count(item_count, part_dt, array[date'2012-01-01']) as first_day,
> intersect_count(item_count, part_dt, array[date'2012-01-02']) as second_day,
> intersect_count(item_count, part_dt, array[date'2012-01-03']) as third_day,
> intersect_count(item_count, part_dt,
> array[date'2012-01-01',date'2012-01-02']) as retention_oneday,
> intersect_count(item_count, part_dt,
> array[date'2012-01-01',date'2012-01-02',date'2012-01-03']) as retention_twoday
> from kylin_sales
> where part_dt in (date'2012-01-01',date'2012-01-02',date'2012-01-03')
> group by PART_DT
> LIMIT 5": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:834)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:561)
> at org.apache.kylin.rest.service.QueryService.query(QueryService.java:181)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:415)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:78)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> ..
> Caused by: java.lang.NullPointerException
> at
> org.roaringbitmap.buffer.MutableRoaringBitmap.or(MutableRoaringBitmap.java:1041)
> at
> org.apache.kylin.measure.bitmap.RoaringBitmapCounter.orWith(RoaringBitmapCounter.java:72)
> at
> org.apache.kylin.measure.bitmap.BitmapIntersectDistinctCountAggFunc$RetentionPartialResult.add(BitmapIntersectDistinctCountAggFunc.java:57)
> at
> org.apache.kylin.measure.bitmap.BitmapIntersectDistinctCountAggFunc.add(BitmapIntersectDistinctCountAggFunc.java:90)
> at Baz$4.apply(ANONYMOUS.java:136)
> at Baz$4.apply(ANONYMOUS.java:158)
> at Baz$4.apply(ANONYMOUS.java)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:832)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
> at Baz.bind(Baz.java:99)
> How to re-produce:
> 1. run $KYLIN_HOME/bin/sample.sh
> 2. then create a cube like
> {
> "uuid": "9554f6f6-74dc-489e-b780-2f48f281576c",
> "last_modified": 1511247707372,
> "version": "2.2.0.0",
> "name": "test",
> "is_draft": false,
> "model_name": "kylin_sales_model",
> "description": "",
> "null_string": null,
> "dimensions": [
> {
> "name": "PART_DT",
> "table": "KYLIN_SALES",
> "column": "PART_DT",
> "derived": null
> },
> {
> "name": "LEAF_CATEG_ID",
> "table": "KYLIN_SALES",
> "column": "LEAF_CATEG_ID",
> "derived": null
> },
> {
> "name": "LSTG_SITE_ID",
> "table": "KYLIN_SALES",
> "column": "LSTG_SITE_ID",
> "derived": null
> },
> {
> "name": "CAL_DT",
> "table": "KYLIN_CAL_DT",
> "column": null,
> "derived": [
> "CAL_DT"
> ]
> },
> {
> "name": "LEAF_CATEG_ID",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
> "LEAF_CATEG_ID"
> ]
> },
> {
> "name": "USER_DEFINED_FIELD1",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
> "USER_DEFINED_FIELD1"
> ]
> },
> {
> "name": "USER_DEFINED_FIELD3",
> "table": "KYLIN_CATEGORY_GROUPINGS",
> "column": null,
> "derived": [
>
[jira] [Updated] (KYLIN-3067) Provide web interface for dimension capping feature
[ https://issues.apache.org/jira/browse/KYLIN-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated KYLIN-3067: - Attachment: 0001-KYLIN-3067-provide-dim-cap-in-GUL.patch I have completed this jira. Please reviewed it.Thanks! [~bob123] > Provide web interface for dimension capping feature > --- > > Key: KYLIN-3067 > URL: https://issues.apache.org/jira/browse/KYLIN-3067 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: Roger Shi >Assignee: Zhixiong Chen > Attachments: 0001-KYLIN-3067-provide-dim-cap-in-GUL.patch > > > As [KYLIN-2363] has provided the dimension capping feature in the backend, we > need a proper web interface. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2771) Turn on all TIMEDATE FUNCTIONS in ODBC
[ https://issues.apache.org/jira/browse/KYLIN-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2771. -- Resolution: Fixed Fix Version/s: v2.2.0 fixed in commit 631fe416cb60223c5f96325fda40abbea24cff3b > Turn on all TIMEDATE FUNCTIONS in ODBC > -- > > Key: KYLIN-2771 > URL: https://issues.apache.org/jira/browse/KYLIN-2771 > Project: Kylin > Issue Type: Improvement >Reporter: Roger Shi > Fix For: v2.2.0 > > > Only 4 functions are declared in ODBC while actually there's no such > limitation in Kylin implementation. Turn them on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2850) Throw clear error message when there're more than 63 columns in rowkey
[ https://issues.apache.org/jira/browse/KYLIN-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2850. -- Resolution: Fixed Fix Version/s: v2.2.0 fixed in fae56af2166ac21b5737542ca4c9ad1fb5b7622b > Throw clear error message when there're more than 63 columns in rowkey > -- > > Key: KYLIN-2850 > URL: https://issues.apache.org/jira/browse/KYLIN-2850 > Project: Kylin > Issue Type: Improvement >Reporter: Roger Shi >Priority: Minor > Fix For: v2.2.0 > > > Now the error message is not that clear when there're more than 63 columns in > rowkey. Provide clearer error message in this case. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2785) Handle Maprfs schema working directory
[ https://issues.apache.org/jira/browse/KYLIN-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2785. -- Resolution: Fixed Assignee: Roger Shi Fix Version/s: v2.2.0 Fixed in commit 5008866f41efd13ce5abaf8cc19c09919fe698dd > Handle Maprfs schema working directory > -- > > Key: KYLIN-2785 > URL: https://issues.apache.org/jira/browse/KYLIN-2785 > Project: Kylin > Issue Type: Improvement >Reporter: Roger Shi >Assignee: Roger Shi >Priority: Minor > Fix For: v2.2.0 > > > Working directory starting with "maprfs" always return "maprfs:/fileXXX/xxx", > even setting "maprfs:///fileXXX/xxx" in kylin.properties file. The behavior > is different from HDFS. Inserting the two missing slash will fix it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3065) Add 'First' and 'Last' button in case 'Query History' is too much
[ https://issues.apache.org/jira/browse/KYLIN-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276233#comment-16276233 ] peng.jianhua commented on KYLIN-3065: - [~Zhixiong Chen],ok,thanks. > Add 'First' and 'Last' button in case 'Query History' is too much > - > > Key: KYLIN-3065 > URL: https://issues.apache.org/jira/browse/KYLIN-3065 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Attachments: > 0001-KYLIN-3065-Add-First-and-Last-button-in-case-Query-H.patch, 01.png > > > When the 'Query History' or 'Saved Queries' is too much.Just 'Previous' and > 'Next' can not meet the demand. > Such as I just want to find my first query and I've searched too much times,I > need to click to much times to find out. > So I add some attributes for the label 'Pagination'. > The following 01.png is the comparison picture. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3076) Make kylin remember the choices we have made in the "Monitor>Jobs" page
[ https://issues.apache.org/jira/browse/KYLIN-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276228#comment-16276228 ] Zhixiong Chen commented on KYLIN-3076: -- Hi，[~peng.jianhua] Please don't put too many value into sessionStorage. Maybe you could create a new object in jobListModel.js to manage it. > Make kylin remember the choices we have made in the "Monitor>Jobs" page > --- > > Key: KYLIN-3076 > URL: https://issues.apache.org/jira/browse/KYLIN-3076 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.3.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Attachments: > 0001-KYLIN-3076-Make-kylin-remember-the-choices-we-have-m.patch > > > In "Monitor>Jobs" page, the defaut setting is to list all the jobs in the > last one week, if we only want to see the jobs whose cube named "cube1" and > was successfully finished in the last one month, we have to type "cube1" in > the filter box, select "LAST ONE MONTH" in the drop-down box, and check the > "FINISHED" item in the job status checkbox, then click the "refresh" button, > the results we want will come out. But, when we leave this page for a moment, > for example, we click "MODEL" menu to see the model infomation and then click > "MONITOR" menu to turn back to the monitor page, we will find that all the > choices we have made was gone, and have to do it again to filter out the jobs > we want. > We hope that kylin can remember the choices we have made in the > "Monitor>Jobs" page, so we don't have to make the same choices everytime when > we enter this page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3061) When we cancel the Topic modification for 'Kafka Setting' of streaming table, the 'Cancel' operation will make a mistake.
[
https://issues.apache.org/jira/browse/KYLIN-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276220#comment-16276220
]
Zhixiong Chen commented on KYLIN-3061:
--
Hi, [~peng.jianhua].
I think the cluster edit is confused.
Maybe you could make the 'add cluster' and 'edit cluster' in same logic.
> When we cancel the Topic modification for 'Kafka Setting' of streaming table,
> the 'Cancel' operation will make a mistake.
> -
>
> Key: KYLIN-3061
> URL: https://issues.apache.org/jira/browse/KYLIN-3061
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.3.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Attachments:
> 0001-KYLIN-3061-When-we-cancel-the-Topic-modification-for.patch,
> can_add_the_same_record.png, cancel_is_not_right.png, edit_kafka_setting.png,
> edit_streaming_table.png
>
>
> There are two bugs in this issue.
> First
> 1. Choose one streaming table, choose the 'Streaming Cluster' tab, then click
> 'Edit' button, refer to [^edit_streaming_table.png]
> 2. The edit page will be opened, then you can edit the value of topic of
> 'Kafka Setting'. As long as you modify the ID, Host, Port value, the original
> value tag will follow changes, refer to [^edit_kafka_setting.png];
> 3. When you click 'cancel' button, you will find the old values have been
> changed to the new values, and if you click the 'submit' button, you will
> also find the values to be canceled will be submitted, refer to
> [^cancel_is_not_right.png]
> But I think the correct way should be that 'cancel' button will not change
> any value.
> Second
> The follow code in streamingConfig.js has a bug, even if "cluster.newBroker"
> object matchs the one element of "cluster.brokers" array, this "if" decision
> will return false. Because even though the "cluster.newBroker" object has the
> same attribute values as an element in the array, their storage addresses may
> be different. The result is that you can add several same record, like
> [^can_add_the_same_record.png].
> {code:java}
> $scope.saveNewBroker = function(cluster) {
> if (cluster.brokers.indexOf(cluster.newBroker) === -1) {
> ..
> }
> }
> {code}
> So I have repaired these two bugs, please check the patch, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-3069) Add proper time zone support to the WebUI instead of GMT/PST kludge
[ https://issues.apache.org/jira/browse/KYLIN-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276213#comment-16276213 ] Zhixiong Chen commented on KYLIN-3069: -- OK.Thanks [~peng.jianhua]. > Add proper time zone support to the WebUI instead of GMT/PST kludge > --- > > Key: KYLIN-3069 > URL: https://issues.apache.org/jira/browse/KYLIN-3069 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.2.0 > Environment: HDP 2.5.3, Kylin 2.2.0 >Reporter: Vsevolod Ostapenko >Assignee: peng.jianhua >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > Time zone handling logic in the WebUI is a kludge, coded to parse only > "GMT-N" time zone specifications and defaulting to PST, if parsing is not > successful (kylin/webapp/app/js/filters/filter.js) > Integrating moment and moment time zone (http://momentjs.com/timezone/docs/) > into the product, would allow correct time zone handling. > For the users who happen to reside in the geographical locations that do > observe day light savings time, usage of GMT-N format is very inconvenient > and info reported by the UI in various places is perplexing. > Needless to say that the GMT moniker itself is long deprecated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-3069) Add proper time zone support to the WebUI instead of GMT/PST kludge
[ https://issues.apache.org/jira/browse/KYLIN-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peng.jianhua reassigned KYLIN-3069: --- Assignee: peng.jianhua (was: Zhixiong Chen) > Add proper time zone support to the WebUI instead of GMT/PST kludge > --- > > Key: KYLIN-3069 > URL: https://issues.apache.org/jira/browse/KYLIN-3069 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.2.0 > Environment: HDP 2.5.3, Kylin 2.2.0 >Reporter: Vsevolod Ostapenko >Assignee: peng.jianhua >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > Time zone handling logic in the WebUI is a kludge, coded to parse only > "GMT-N" time zone specifications and defaulting to PST, if parsing is not > successful (kylin/webapp/app/js/filters/filter.js) > Integrating moment and moment time zone (http://momentjs.com/timezone/docs/) > into the product, would allow correct time zone handling. > For the users who happen to reside in the geographical locations that do > observe day light savings time, usage of GMT-N format is very inconvenient > and info reported by the UI in various places is perplexing. > Needless to say that the GMT moniker itself is long deprecated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3069) Add proper time zone support to the WebUI instead of GMT/PST kludge
[ https://issues.apache.org/jira/browse/KYLIN-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276210#comment-16276210 ] peng.jianhua commented on KYLIN-3069: - Hi [~Shaofengshi] and [~Zhixiong Chen], I will fix the issue. Thanks! > Add proper time zone support to the WebUI instead of GMT/PST kludge > --- > > Key: KYLIN-3069 > URL: https://issues.apache.org/jira/browse/KYLIN-3069 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.2.0 > Environment: HDP 2.5.3, Kylin 2.2.0 >Reporter: Vsevolod Ostapenko >Assignee: Zhixiong Chen >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > Time zone handling logic in the WebUI is a kludge, coded to parse only > "GMT-N" time zone specifications and defaulting to PST, if parsing is not > successful (kylin/webapp/app/js/filters/filter.js) > Integrating moment and moment time zone (http://momentjs.com/timezone/docs/) > into the product, would allow correct time zone handling. > For the users who happen to reside in the geographical locations that do > observe day light savings time, usage of GMT-N format is very inconvenient > and info reported by the UI in various places is perplexing. > Needless to say that the GMT moniker itself is long deprecated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3060) The logical processing of creating or updating streaming table has a bug in server, which will cause a NullPointerException.
[
https://issues.apache.org/jira/browse/KYLIN-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276194#comment-16276194
]
peng.jianhua commented on KYLIN-3060:
-
Hi [~liyang.g...@gmail.com], I'm sorry, I will pay attention to the code
format, thanks for your suggestion.
> The logical processing of creating or updating streaming table has a bug in
> server, which will cause a NullPointerException.
>
>
> Key: KYLIN-3060
> URL: https://issues.apache.org/jira/browse/KYLIN-3060
> Project: Kylin
> Issue Type: Bug
> Components: streaming
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Fix For: v2.3.0
>
> Attachments:
> 0001-KYLIN-3060-The-logical-processing-of-creating-or-upd.patch,
> after_repaired.png, create_non_interger_ID.png, edit_non_interger_ID.png
>
>
> When you set the invalid parameter for creating or updating a streaming
> table, the client will show an error message dialog, refer to
> [^create_non_interger_ID.png] and [^edit_non_interger_ID.png].
> I analyze the Kylin server log, there are some null pointer exceptions, then
> I analyze the StreamingController.java, whose logical processing of streaming
> table has a bug, which will cause a NullPointerException.
> eg. When the streamingRequest.getKafkaConfig() has a invalid pamamter, the
> kafkaConfig will be null, then "if (desc.getUuid() == null || desc.getName()
> == null) " in function
> "org.apache.kylin.source.kafka.KafkaConfigManager.updateKafkaConfig" will
> cause the NullPointerException.
> {code:java}
> public StreamingRequest updateStreamingConfig(@RequestBody StreamingRequest
> streamingRequest) throws JsonProcessingException {
> ..
> KafkaConfig kafkaConfig = deserializeKafkaSchemalDesc(streamingRequest);
> ..
> kafkaConfig = kafkaConfigService.updateKafkaConfig(kafkaConfig, project);
> ..
> }
> private KafkaConfig deserializeKafkaSchemalDesc(StreamingRequest
> streamingRequest) {
> KafkaConfig desc = null;
> try {
> logger.debug("Saving KafkaConfig " +
> streamingRequest.getKafkaConfig());
> desc = JsonUtil.readValue(streamingRequest.getKafkaConfig(),
> KafkaConfig.class);
> } catch (JsonParseException e) {
> logger.error("The KafkaConfig definition is invalid.", e);
> updateRequest(streamingRequest, false, e.getMessage());
> } catch (JsonMappingException e) {
> logger.error("The data KafkaConfig definition is invalid.", e);
> updateRequest(streamingRequest, false, e.getMessage());
> } catch (IOException e) {
> logger.error("Failed to deal with the request.", e);
> throw new InternalErrorException("Failed to deal with the
> request:" + e.getMessage(), e);
> }
> return desc;
> }
> public StreamingConfig updateStreamingConfig(StreamingConfig config,
> String project) throws IOException {
> aclEvaluate.checkProjectAdminPermission(project);
> return getStreamingManager().updateStreamingConfig(config);
> }
> public StreamingConfig updateStreamingConfig(StreamingConfig desc) throws
> IOException {
> // Validate CubeDesc
> if (desc.getUuid() == null || desc.getName() == null) {
> ..
>}
> {code}
> So I have repaired it, then the client will tell you the detail error
> message( refer to [^after_repaired.png]), the server no longer throws an null
> point exception.
> you can check the patch, thanks!
> {panel:title=Kylin server log}
> 2017-11-24 18:38:00,702 ERROR [http-bio-7070-exec-14]
> controller.StreamingController:202 : Failed to deal with the request:null
> {color:red}java.lang.NullPointerException{color}
> at
> org.apache.kylin.source.kafka.KafkaConfigManager.updateKafkaConfig(KafkaConfigManager.java:156)
> at
> org.apache.kylin.rest.service.KafkaConfigService.updateKafkaConfig(KafkaConfigService.java:84)
> {color:red}at
> org.apache.kylin.rest.controller.StreamingController.updateStreamingConfig(StreamingController.java:198){color}
> {panel}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-1664) rest api '/kylin/api/admin/config' without security check
[
https://issues.apache.org/jira/browse/KYLIN-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275911#comment-16275911
]
Shaofeng SHI commented on KYLIN-1664:
-
[~steven.li] Hanhui, are you working on this? Welcome to contribute a patch to
Kylin.
> rest api '/kylin/api/admin/config' without security check
> -
>
> Key: KYLIN-1664
> URL: https://issues.apache.org/jira/browse/KYLIN-1664
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
> Environment: Ubuntu 14.4
> Jdk 1.7.0
> Kylin 1.5.1 binary
>Reporter: Hanhui LI
>Assignee: hongbin ma
> Labels: test
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> rest api '/kylin/api/admin/config' without security check.
> Please check the follwoing:
> ===
> GET Request:
> http://127.0.0.1:7070/kylin/api/admin/config
> Response:
> {"config":"kylin.hbase.region.cut.large=50\nkylin.hbase.default.compression.codec=snappy\ndeploy.env=QA\nacl.adminRole=ROLE_ADMIN\nkylin.sandbox=true\nkylin.hdfs.working.dir=/kylin\nldap.user.searchBase=\nkylin.job.concurrent.max.limit=10\nkylin.job.remote.cli.password=\nsaml.metadata.file=classpath:sso_metadata.xml\nkylin.job.yarn.app.rest.check.interval.seconds=10\nmail.sender=\nmail.password=\nkylin.job.remote.cli.username=\nmail.username=\nsaml.context.serverPort=443\nkylin.web.help.length=4\nkylin.job.run.as.remote.cmd=false\nldap.service.searchPattern=\nkylin.web.contact_mail=\nldap.user.groupSearchBase=\nkylin.hbase.region.cut.small=5\nkylin.web.hive.limit=20\nkylin.job.mapreduce.default.reduce.input.mb=500\nkylin.job.hive.database.for.intermediatetable=default\nkylin.metadata.url=kylin_metadata@hbase\nldap.password=\nldap.username=\nkylin.storage.url=hbase\nganglia.port=8664\nldap.user.searchPattern=\nkylin.job.status.with.kerberos=false\nganglia.group=\nkylin.hbase.cluster.fs=\nacl.defaultRole=ROLE_ANALYST,ROLE_MODELER\nsaml.context.contextPath=/kylin\nmail.host=\nkylin.job.remote.cli.working.dir=/tmp/kylin\nkylin.web.diagnostic=\nsaml.context.scheme=https\nkylin.job.cubing.inmem.sampling.percent=100\nldap.service.groupSearchBase=\nsaml.metadata.entityBaseURL=https://hostname/kylin\nkylin.hbase.hfile.size.gb=5\nldap.service.searchBase=\nkylin.owner=who...@kylin.apache.org\nmail.enabled=false\nkylin.rest.servers=localhost:7070\nkylin.security.profile=testing\nkylin.job.retry=0\nsaml.context.serverName=hostname\nldap.server=ldap://ldap_server:389\nkylin.job.remote.cli.hostname=\nkylin.query.security.enabled=true\nkylin.server.mode=all\nkylin.web.help.3=onboard|Cube
> Design Tutorial|\nkylin.web.help.2=tableau|Tableau
> Guide|\nkylin.web.help.1=odbc|ODBC
> Driver|\nkylin.hbase.region.cut.medium=10\nkylin.web.help.0=start|Getting
> Started|\nkylin.web.hadoop=\nkylin.web.streaming.guide=http://kylin.apache.org/\n"}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2736) Use multiple threads to calculate HyperLogLogPlusCounter in FactDistinctColumnsMapper
[ https://issues.apache.org/jira/browse/KYLIN-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2736: -- Fix Version/s: v2.3.0 > Use multiple threads to calculate HyperLogLogPlusCounter in > FactDistinctColumnsMapper > - > > Key: KYLIN-2736 > URL: https://issues.apache.org/jira/browse/KYLIN-2736 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2866) Enlarge the reducer number for hyperloglog statistics calculation at step FactDistinctColumnsJob
[ https://issues.apache.org/jira/browse/KYLIN-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2866: -- Fix Version/s: v2.3.0 > Enlarge the reducer number for hyperloglog statistics calculation at step > FactDistinctColumnsJob > > > Key: KYLIN-2866 > URL: https://issues.apache.org/jira/browse/KYLIN-2866 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Currently only one reducer is assigned for hll stats calculation, which may > become the bottleneck for slow down this step. Since the stats for different > cuboids will not influence each other, it's better to divide the cuboid set > into several and assign a reduce for each subset. > The strategy of this patch is to assign 100 cuboids into a subset. And > there's a upper limit of reducers for hll stats calculation. Currently it's > 50. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2726) Introduce a dashboard for showing kylin service related metrics, like query count, query latency, job count, etc
[ https://issues.apache.org/jira/browse/KYLIN-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275910#comment-16275910 ] Zhong Yanghong commented on KYLIN-2726: --- https://github.com/apache/kylin/pull/85 > Introduce a dashboard for showing kylin service related metrics, like query > count, query latency, job count, etc > > > Key: KYLIN-2726 > URL: https://issues.apache.org/jira/browse/KYLIN-2726 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Pan, Julian > Fix For: v2.3.0 > > > Dashboard shows query count, query latency, job count with many dimensions, > like server level, project level, cube level, different time granularity -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2727) Introduce cube planner able to select cost-effective cuboids to be built by cost-based algorithms
[ https://issues.apache.org/jira/browse/KYLIN-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2727. --- Resolution: Resolved > Introduce cube planner able to select cost-effective cuboids to be built by > cost-based algorithms > - > > Key: KYLIN-2727 > URL: https://issues.apache.org/jira/browse/KYLIN-2727 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > There're several disadvantages to create partial cubes only based on static > rules: > * To learn the concept of these rules will bring extra burden to cube admins > * To achieve a goal cuboid set may need a very complicated set of static rules > * Cube designers may be not familiar with business related query patterns, > resulting in few static rules able to applied > * Static rules created at first may not correct or user query patterns are > changing dynamically > The goal of cube planner is to reduce effort for cube admins to design an > effective partial cube. It owns the following advantages: > * No sophisticated combination of static rules is needed > * Allow more dimensions to be included with few static rules > * Able to adjust a non-cost-effective cuboid set to a cost-effective one > based on historical queries -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong resolved KYLIN-2735.
---
Resolution: Resolved
Fix Version/s: v2.3.0
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** {color:#f79232}first check the job priority queue and schedule jobs
> meeting condition{color} {color:#f691b2}*C*{color}
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition {color:#f691b2}*C*{color} is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2734. --- Resolution: Resolved > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase and it is able to > go through cube planner algorithm to get recommended cuboids, regardless > whether the total number of defined cuboids is less than the threshold to > trigger cube planner phase one, the cube planner phase one will be triggered > and these imported cuboids will be regarded as mandatory cuboids for cube > planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147022#comment-16147022 ] Zhong Yanghong edited comment on KYLIN-2734 at 12/3/17 12:13 PM: - https://github.com/apache/kylin/pull/85 was (Author: yaho): Related commit for branch yaho-cube-planner is as follows: * Backend: https://github.com/apache/kylin/commit/7279427d40df7e9d6151ecf3f439c2c20f800dd1 > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase and it is able to > go through cube planner algorithm to get recommended cuboids, regardless > whether the total number of defined cuboids is less than the threshold to > trigger cube planner phase one, the cube planner phase one will be triggered > and these imported cuboids will be regarded as mandatory cuboids for cube > planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2734: -- Fix Version/s: v2.3.0 > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase and it is able to > go through cube planner algorithm to get recommended cuboids, regardless > whether the total number of defined cuboids is less than the threshold to > trigger cube planner phase one, the cube planner phase one will be triggered > and these imported cuboids will be regarded as mandatory cuboids for cube > planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2734: -- Affects Version/s: (was: v2.1.0) > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase and it is able to > go through cube planner algorithm to get recommended cuboids, regardless > whether the total number of defined cuboids is less than the threshold to > trigger cube planner phase one, the cube planner phase one will be triggered > and these imported cuboids will be regarded as mandatory cuboids for cube > planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2902) Introduce project-level query number control
[ https://issues.apache.org/jira/browse/KYLIN-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2902: -- Fix Version/s: v2.3.0 > Introduce project-level query number control > > > Key: KYLIN-2902 > URL: https://issues.apache.org/jira/browse/KYLIN-2902 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > Attachments: APACHE-KYLIN-2902.patch > > > One kylin server may contain many projects. To avoid some projects occupying > too much resources, like http connection, project-level query number limit is > useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3043) Don't need create materialized view for lookup tables without snapshot
[ https://issues.apache.org/jira/browse/KYLIN-3043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-3043: -- Fix Version/s: v2.3.0 > Don't need create materialized view for lookup tables without snapshot > -- > > Key: KYLIN-3043 > URL: https://issues.apache.org/jira/browse/KYLIN-3043 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > Attachments: APACHE-KYLIN-3043.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[
https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2731:
--
Fix Version/s: v2.3.0
> Introduce checkpoint executable
> ---
>
> Key: KYLIN-2731
> URL: https://issues.apache.org/jira/browse/KYLIN-2731
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
> Attachments: One-Click Cube Optimization.png
>
>
> The basic idea of cube optimization without influencing current queries is
> shown at figure [^One-Click Cube Optimization.png].
> To optimize a cube with N ready segments,
> * For each segment, a corresponding optimize job will be triggered to adjust
> its cuboids.
> * If a optimize job finished, it's resulting segment status should be
> READY_PENDING and will not serve query.
> * There should be job to wait for all of the optimize jobs finishing their
> work, and then atomically remove old READY segments and update the status of
> READY_PENDING segments to READY for serving query. Here, we call this kind of
> job as {color:#f79232}checkpoint executable{color}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[
https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2731:
--
Affects Version/s: (was: v2.1.0)
> Introduce checkpoint executable
> ---
>
> Key: KYLIN-2731
> URL: https://issues.apache.org/jira/browse/KYLIN-2731
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
> Attachments: One-Click Cube Optimization.png
>
>
> The basic idea of cube optimization without influencing current queries is
> shown at figure [^One-Click Cube Optimization.png].
> To optimize a cube with N ready segments,
> * For each segment, a corresponding optimize job will be triggered to adjust
> its cuboids.
> * If a optimize job finished, it's resulting segment status should be
> READY_PENDING and will not serve query.
> * There should be job to wait for all of the optimize jobs finishing their
> work, and then atomically remove old READY segments and update the status of
> READY_PENDING segments to READY for serving query. Here, we call this kind of
> job as {color:#f79232}checkpoint executable{color}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2731) Introduce checkpoint executable
[
https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275909#comment-16275909
]
Zhong Yanghong commented on KYLIN-2731:
---
https://github.com/apache/kylin/pull/85
> Introduce checkpoint executable
> ---
>
> Key: KYLIN-2731
> URL: https://issues.apache.org/jira/browse/KYLIN-2731
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.3.0
>
> Attachments: One-Click Cube Optimization.png
>
>
> The basic idea of cube optimization without influencing current queries is
> shown at figure [^One-Click Cube Optimization.png].
> To optimize a cube with N ready segments,
> * For each segment, a corresponding optimize job will be triggered to adjust
> its cuboids.
> * If a optimize job finished, it's resulting segment status should be
> READY_PENDING and will not serve query.
> * There should be job to wait for all of the optimize jobs finishing their
> work, and then atomically remove old READY segments and update the status of
> READY_PENDING segments to READY for serving query. Here, we call this kind of
> job as {color:#f79232}checkpoint executable{color}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2822: -- Fix Version/s: v2.3.0 > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Pan, Julian > Fix For: v2.3.0 > > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2843) Upgrade nvd3 version
[ https://issues.apache.org/jira/browse/KYLIN-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2843: -- Affects Version/s: (was: v2.2.0) > Upgrade nvd3 version > > > Key: KYLIN-2843 > URL: https://issues.apache.org/jira/browse/KYLIN-2843 > Project: Kylin > Issue Type: Task > Components: Web >Reporter: Pan, Julian >Assignee: Pan, Julian > Fix For: v2.3.0 > > > 1. Upgrade nvd3 version for sunburst chart in cube planner. > 2. Update related chart for insight page and dashboard page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2822: -- Affects Version/s: (was: v2.1.0) > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Pan, Julian > Fix For: v2.3.0 > > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2843) Upgrade nvd3 version
[ https://issues.apache.org/jira/browse/KYLIN-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2843: -- Fix Version/s: v2.3.0 > Upgrade nvd3 version > > > Key: KYLIN-2843 > URL: https://issues.apache.org/jira/browse/KYLIN-2843 > Project: Kylin > Issue Type: Task > Components: Web >Reporter: Pan, Julian >Assignee: Pan, Julian > Fix For: v2.3.0 > > > 1. Upgrade nvd3 version for sunburst chart in cube planner. > 2. Update related chart for insight page and dashboard page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2843) Upgrade nvd3 version
[ https://issues.apache.org/jira/browse/KYLIN-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275908#comment-16275908 ] Zhong Yanghong commented on KYLIN-2843: --- https://github.com/apache/kylin/pull/85 > Upgrade nvd3 version > > > Key: KYLIN-2843 > URL: https://issues.apache.org/jira/browse/KYLIN-2843 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v2.2.0 >Reporter: Pan, Julian >Assignee: Pan, Julian > > 1. Upgrade nvd3 version for sunburst chart in cube planner. > 2. Update related chart for insight page and dashboard page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2822: -- Comment: was deleted (was: Related commit for branch yaho-cube-planner is as follows: * Backend: https://github.com/apache/kylin/commit/0b12df776348c7863bea6363ceacfe7380cb4e15) > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Pan, Julian > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2843) Upgrade nvd3 version
[ https://issues.apache.org/jira/browse/KYLIN-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2843. --- Resolution: Resolved > Upgrade nvd3 version > > > Key: KYLIN-2843 > URL: https://issues.apache.org/jira/browse/KYLIN-2843 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v2.2.0 >Reporter: Pan, Julian >Assignee: Pan, Julian > > 1. Upgrade nvd3 version for sunburst chart in cube planner. > 2. Update related chart for insight page and dashboard page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275905#comment-16275905 ] Zhong Yanghong commented on KYLIN-2732: --- https://github.com/apache/kylin/pull/85 > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275906#comment-16275906 ] Zhong Yanghong commented on KYLIN-2822: --- https://github.com/apache/kylin/pull/85 > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Pan, Julian > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2733: -- Comment: was deleted (was: Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/0c4b3ad57277653e2b360c0acbf1c9c2b6d4d6f2) > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275904#comment-16275904 ] Zhong Yanghong commented on KYLIN-2733: --- https://github.com/apache/kylin/pull/85 > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3018) Change maxLevel for layered cubing
[ https://issues.apache.org/jira/browse/KYLIN-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275902#comment-16275902 ] Zhong Yanghong commented on KYLIN-3018: --- https://github.com/apache/kylin/pull/85 > Change maxLevel for layered cubing > -- > > Key: KYLIN-3018 > URL: https://issues.apache.org/jira/browse/KYLIN-3018 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2822. --- Resolution: Resolved > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Pan, Julian > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2733. --- Resolution: Resolved > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2733: -- Fix Version/s: v2.3.0 > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2733: -- Affects Version/s: (was: v2.1.0) > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-3018) Change maxLevel for layered cubing
[ https://issues.apache.org/jira/browse/KYLIN-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-3018. --- Resolution: Resolved > Change maxLevel for layered cubing > -- > > Key: KYLIN-3018 > URL: https://issues.apache.org/jira/browse/KYLIN-3018 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2731) Introduce checkpoint executable
[
https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong resolved KYLIN-2731.
---
Resolution: Resolved
> Introduce checkpoint executable
> ---
>
> Key: KYLIN-2731
> URL: https://issues.apache.org/jira/browse/KYLIN-2731
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.1.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: One-Click Cube Optimization.png
>
>
> The basic idea of cube optimization without influencing current queries is
> shown at figure [^One-Click Cube Optimization.png].
> To optimize a cube with N ready segments,
> * For each segment, a corresponding optimize job will be triggered to adjust
> its cuboids.
> * If a optimize job finished, it's resulting segment status should be
> READY_PENDING and will not serve query.
> * There should be job to wait for all of the optimize jobs finishing their
> work, and then atomically remove old READY segments and update the status of
> READY_PENDING segments to READY for serving query. Here, we call this kind of
> job as {color:#f79232}checkpoint executable{color}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275899#comment-16275899 ] Zhong Yanghong commented on KYLIN-2732: --- Both raw data & base cuboid data can be cubing source. > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2732: -- Affects Version/s: (was: v2.1.0) > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2732: -- Fix Version/s: v2.3.0 > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.3.0 > > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2732. --- Resolution: Resolved > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.1.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2726) Introduce a dashboard for showing kylin service related metrics, like query count, query latency, job count, etc
[ https://issues.apache.org/jira/browse/KYLIN-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2726: -- Fix Version/s: v2.3.0 > Introduce a dashboard for showing kylin service related metrics, like query > count, query latency, job count, etc > > > Key: KYLIN-2726 > URL: https://issues.apache.org/jira/browse/KYLIN-2726 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Pan, Julian > Fix For: v2.3.0 > > > Dashboard shows query count, query latency, job count with many dimensions, > like server level, project level, cube level, different time granularity -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2726) Introduce a dashboard for showing kylin service related metrics, like query count, query latency, job count, etc
[ https://issues.apache.org/jira/browse/KYLIN-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong resolved KYLIN-2726. --- Resolution: Resolved > Introduce a dashboard for showing kylin service related metrics, like query > count, query latency, job count, etc > > > Key: KYLIN-2726 > URL: https://issues.apache.org/jira/browse/KYLIN-2726 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Pan, Julian > > Dashboard shows query count, query latency, job count with many dimensions, > like server level, project level, cube level, different time granularity -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2726) Introduce a dashboard for showing kylin service related metrics, like query count, query latency, job count, etc
[ https://issues.apache.org/jira/browse/KYLIN-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2726: -- Affects Version/s: (was: v2.0.0) > Introduce a dashboard for showing kylin service related metrics, like query > count, query latency, job count, etc > > > Key: KYLIN-2726 > URL: https://issues.apache.org/jira/browse/KYLIN-2726 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Pan, Julian > Fix For: v2.3.0 > > > Dashboard shows query count, query latency, job count with many dimensions, > like server level, project level, cube level, different time granularity -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2902) Introduce project-level query number control
[ https://issues.apache.org/jira/browse/KYLIN-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2902: -- Attachment: APACHE-KYLIN-2902.patch > Introduce project-level query number control > > > Key: KYLIN-2902 > URL: https://issues.apache.org/jira/browse/KYLIN-2902 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: APACHE-KYLIN-2902.patch > > > One kylin server may contain many projects. To avoid some projects occupying > too much resources, like http connection, project-level query number limit is > useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2902) Introduce project-level query number control
[ https://issues.apache.org/jira/browse/KYLIN-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2902: -- Attachment: (was: APACHE-KYLIN-2902.patch) > Introduce project-level query number control > > > Key: KYLIN-2902 > URL: https://issues.apache.org/jira/browse/KYLIN-2902 > Project: Kylin > Issue Type: New Feature >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > One kylin server may contain many projects. To avoid some projects occupying > too much resources, like http connection, project-level query number limit is > useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2974) zero joint group can lead to query error
[
https://issues.apache.org/jira/browse/KYLIN-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyang resolved KYLIN-2974.
---
Resolution: Fixed
Fix Version/s: v2.3.0
> zero joint group can lead to query error
>
>
> Key: KYLIN-2974
> URL: https://issues.apache.org/jira/browse/KYLIN-2974
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.1.0
>Reporter: Yang Hao
>Assignee: Yang Hao
> Fix For: v2.3.0
>
> Attachments: KYLIN-2974.patch
>
>
> When query a sql, it turns to find all possible aggregation group. But it may
> cause error when the joint group is empty. Relevant code is
> {code:java}
> cuboidID = cuboidID | Collections.min(agg.getJoints(),
> cuboidSelectComparator);
> {code}
> In our env, when we have multiple aggregation groups, it may trigger the error
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2974) zero joint group can lead to query error
[
https://issues.apache.org/jira/browse/KYLIN-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275854#comment-16275854
]
liyang commented on KYLIN-2974:
---
I understand. Thanks Hao. Commit merged. Please use better commit message next
time. The message was just "init" in the patch and I have revised it.
https://github.com/apache/kylin/commit/f5e22801704b31820f08ca1cae74ab685ec42ec0
> zero joint group can lead to query error
>
>
> Key: KYLIN-2974
> URL: https://issues.apache.org/jira/browse/KYLIN-2974
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.1.0
>Reporter: Yang Hao
>Assignee: Yang Hao
> Attachments: KYLIN-2974.patch
>
>
> When query a sql, it turns to find all possible aggregation group. But it may
> cause error when the joint group is empty. Relevant code is
> {code:java}
> cuboidID = cuboidID | Collections.min(agg.getJoints(),
> cuboidSelectComparator);
> {code}
> In our env, when we have multiple aggregation groups, it may trigger the error
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-3077) EDW.TEST_SELLER_TYPE_DIM_TABLE is not being created by the integration test, but it's presence in the Hive is expected
[ https://issues.apache.org/jira/browse/KYLIN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang resolved KYLIN-3077. --- Resolution: Fixed Fix Version/s: v2.3.0 > EDW.TEST_SELLER_TYPE_DIM_TABLE is not being created by the integration test, > but it's presence in the Hive is expected > -- > > Key: KYLIN-3077 > URL: https://issues.apache.org/jira/browse/KYLIN-3077 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v2.3.0 > Environment: HDP 2.5.6, Kylin 2.3.0 >Reporter: Vsevolod Ostapenko >Assignee: liyang > Fix For: v2.3.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > Attempt to run Kylin 2.3.0 integration tests against clean single node > cluster under HDP 2.5.6 fails at the step "CREATE VIEW > edw.test_seller_type_dim AS SELECT * FROM edw.test_seller_type_dim_table". > Underlying table edw.test_seller_type_dim_table is never being created by any > of the prior steps. > Replication steps: > 1. Install clean HDP 2.5.6 single node cluster > 2. Configure Kylin dev box to use remote cluster created in the step 1. > 3. Run > mvn verify -fae -Dhdp.version=2.5.6.0-40 -P sandbox 2>&1 | tee > ../kylin-verify.log > There will be a failure reported at the Integration test phase. > 4. grep -i -E "create[ ]+(table|view)" ../kylin-verify.log > There is no create table for edw.test_seller_type_dim_table in the list of > previously executed commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-3077) EDW.TEST_SELLER_TYPE_DIM_TABLE is not being created by the integration test, but it's presence in the Hive is expected
[ https://issues.apache.org/jira/browse/KYLIN-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275853#comment-16275853 ] liyang commented on KYLIN-3077: --- Commit: https://github.com/apache/kylin/commit/43d02f4730f17bde981582d5a94591ca9f2c1f22 > EDW.TEST_SELLER_TYPE_DIM_TABLE is not being created by the integration test, > but it's presence in the Hive is expected > -- > > Key: KYLIN-3077 > URL: https://issues.apache.org/jira/browse/KYLIN-3077 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v2.3.0 > Environment: HDP 2.5.6, Kylin 2.3.0 >Reporter: Vsevolod Ostapenko >Assignee: liyang > Original Estimate: 168h > Remaining Estimate: 168h > > Attempt to run Kylin 2.3.0 integration tests against clean single node > cluster under HDP 2.5.6 fails at the step "CREATE VIEW > edw.test_seller_type_dim AS SELECT * FROM edw.test_seller_type_dim_table". > Underlying table edw.test_seller_type_dim_table is never being created by any > of the prior steps. > Replication steps: > 1. Install clean HDP 2.5.6 single node cluster > 2. Configure Kylin dev box to use remote cluster created in the step 1. > 3. Run > mvn verify -fae -Dhdp.version=2.5.6.0-40 -P sandbox 2>&1 | tee > ../kylin-verify.log > There will be a failure reported at the Integration test phase. > 4. grep -i -E "create[ ]+(table|view)" ../kylin-verify.log > There is no create table for edw.test_seller_type_dim_table in the list of > previously executed commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
