[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205374#comment-16205374 ] kangkaisen commented on KYLIN-2180: --- HI, julian. I don't see anywhere I change the ACL in this patch. Could you point out the concrete code ？ > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v2.0.0 > > Attachments: KYLIN-2180-refactor-ProjectRequest.patch, > KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205363#comment-16205363 ] Pan, Julian commented on KYLIN-2180: @kangkaisen I found there is a ACL change for create project in your patch. We allow user create project before your refactor. Do you think we should keep the same behavior with previous? Or can you share me the reason to change it? > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v2.0.0 > > Attachments: KYLIN-2180-refactor-ProjectRequest.patch, > KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769338#comment-15769338 ] Zhong,Jason commented on KYLIN-2180: [~kangkaisen] sorry, my fault. it works fine. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.6.1 > > Attachments: KYLIN-2180-refactor-ProjectRequest.patch, > KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15769318#comment-15769318 ] Zhong,Jason commented on KYLIN-2180: [~kangkaisen] Can't create project on GUI, did you miss any code on GUI? ProjectController.saveProject(@RequestBody ProjectRequest projectRequest) the parameter from GUI is different from property of ProjectRequest > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.6.1 > > Attachments: KYLIN-2180-refactor-ProjectRequest.patch, > KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15753308#comment-15753308 ] liyang commented on KYLIN-2180: --- Thanks Kaisen!! The refactor patch is very good. Merged to master. :-) > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.6.1 > > Attachments: KYLIN-2180-refactor-ProjectRequest.patch, > KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739634#comment-15739634 ] kangkaisen commented on KYLIN-2180: --- HI, [~liyang.g...@gmail.com]. The commit c31c8490b05 has a issue: rename the attribute to "override_kylin_properties" will make front-end get the project override_kylin_properties failed. I fixed this issued and refactored the `ProjectRequest` at the same time. Please you review the patch, Thank you. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.6.1 > > Attachments: KYLIN-2180-tmp.patch, KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732236#comment-15732236 ] kangkaisen commented on KYLIN-2180: --- OK，thank you! > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.6.1 > > Attachments: KYLIN-2180-tmp.patch, KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732227#comment-15732227 ] liyang commented on KYLIN-2180: --- Merged! Many thanks to Kaisen! Did a minor revision commit: https://github.com/apache/kylin/commit/c31c8490b05d5f9618464f431cc3d923b012e9b8 Renamed the attribute to "override_kylin_properties", keep consistent with CubeDesc and other naming conventions. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch, KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727853#comment-15727853 ] kangkaisen commented on KYLIN-2180: --- Hi，[~liyang.g...@gmail.com]，please review this patch when you have time. Thank you. The front-end code of this patch should be OK and has run a long time in our production env. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch, KYLIN-2180.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714131#comment-15714131 ] kangkaisen commented on KYLIN-2180: --- OK, I see. I will do this work continue. Thank you very much. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714125#comment-15714125 ] kangkaisen commented on KYLIN-2180: --- yes. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714121#comment-15714121 ] liyang commented on KYLIN-2180: --- Then we can take below steps to create the cube: - Save cube - Update project - Reload the cube again > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15714109#comment-15714109
]
liyang commented on KYLIN-2180:
---
I searched where {{moveRealizationToProject}} is called, guess you are talking
about creating a new cube?
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713953#comment-15713953
]
kangkaisen commented on KYLIN-2180:
---
I know what you mean.
But the Cube add to Project is after the CubeDesc init, So {{findProjects}} in
{{CubeDesc.init()}} will return null.
In other word，{{moveRealizationToProject}} is a prerequisite of
{{findProjects}}. what do you think of it?
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713916#comment-15713916
]
liyang commented on KYLIN-2180:
---
Try it with a real cube. Test cubes does not hold the same name with desc. Also
{{findProject}} returns a list, but in reality a cube can only belong one
project.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713912#comment-15713912 ] liyang commented on KYLIN-2180: --- See https://github.com/apache/kylin/commit/9fc8f5e207b5111fffaec3335efe0c8947353cfa It's demo of getting owner project in CubeDesc.init() where cube level config is constructed. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713890#comment-15713890
]
liyang commented on KYLIN-2180:
---
Confused... why {{moveRealizationToProject}} is a prerequisite of
{{findProjects}}?
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713876#comment-15713876 ] liyang commented on KYLIN-2180: --- Let me try to decouple the loading process of CubeDesc and Project. Add a project field to CubeDesc seems the simplest solution, but it has backward compatibility issue (how to upgrade old cubes?). Also a bi-way relationship is harder to maintain. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703938#comment-15703938
]
liyang commented on KYLIN-2180:
---
Since KYLIN-2220 is now done. Find a project by CubeDesc is already very
straightforward.
{{ProjectManager.findProjects(CUBE, cubeName)}}
Shall give navigation from cubeDesc to project. Why bother so much work?
If you think there is room of improvement around CubeManager and
CubeDescManager, we can open a new topic to discuss.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15700689#comment-15700689 ] kangkaisen commented on KYLIN-2180: --- if we add project id to cubeDesc metadata, this work may be more easier. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15699198#comment-15699198 ] Dayue Gao commented on KYLIN-2180: -- The problem here is we don't know which project a CubeDesc belongs to when reload CubeDesc. Since there is a one-to-one relationship between CubeInstance and CubeDesc after KYLIN-2220, I propose the following changes to how we manage and sync cube metadata 1) Merge CubeDescManager and CubeManager. I see no need to manage them separately. * Create Cube: first create cube desc locally, then create cube and broadcast CUBE CREATE event, finally update project instance * Update Cube: first update cube desc locally, then broadcast CUBE UPDATE event * Drop Cube: first drop cube desc locally, then drop cube and broadcast CUBE DELETE event, finally update project instance 2) Inside CUBE CREATE/UPDATE listener, reload cube desc and cube from metadata 3) Use "project:cube" as key for cube event. As as result, we can get project config overrides when reloading cube desc. 4) Make cube_desc a nested JSON field of cube (future) [~liyang.g...@gmail.com], [~Shaofengshi], Does it sounds reasonable to you? > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682723#comment-15682723 ] liyang commented on KYLIN-2180: --- Created KYLIN-2220 for the ask. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682706#comment-15682706
]
kangkaisen commented on KYLIN-2180:
---
I agree with you. I think the {{getCubeNameFromDesc}} method is unnecessary.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682701#comment-15682701 ] liyang commented on KYLIN-2180: --- The simplest solution is to enforce same name between Cube & CubeDesc. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682696#comment-15682696
]
liyang commented on KYLIN-2180:
---
Let's fix the navigation from CubeDesc to Project first. Then reusing
{{KylinConfigExt}} will become very straightforward.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682486#comment-15682486 ] kangkaisen commented on KYLIN-2180: --- This idea has another issue: after migrating the cube form staging env to prod env, the cube should inherit the config of prod env project . So I think we should not depend on the front-end to realize this feature. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682274#comment-15682274 ] kangkaisen commented on KYLIN-2180: --- This idea is more simpler and the front-end need more work. But whether if user should get the project config firstly when user create the CubeDesc with JSON directly. at the same time, we need to check the cube config contain the project config in back-end. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682062#comment-15682062 ] liyang commented on KYLIN-2180: --- The navigate from Cube/CubeDesc to Project is not convenient. We can fix that if you need it. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682056#comment-15682056 ] liyang commented on KYLIN-2180: --- Or another (perhaps even simpler) idea is "project level config" is just a batch operation of cube configs. Let "project level config" be the common config of all cubes in the project. We just provide a GUI that can change config of all cubes at once. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676470#comment-15676470
]
kangkaisen commented on KYLIN-2180:
---
Thanks for your review and suggestion.
Your idea is my firstly solution. I gave up this solution because when
{{loadCubeDesc}} we could not know which project is related with the cubeDesc
in every case, besides the code change is not straightforward.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676431#comment-15676431
]
liyang commented on KYLIN-2180:
---
Agree with the use case. However I prefer a different class design, that is to
reuse the current KylinConfigExt (instead of create a new Config class).
Idea is to maintain a project level {{Map overrideKylinProps}},
similar to the cube level. In {{CubeDesc.init()}} where cube config
initialized, create the {{overrideKylinProps}} by overlaying cube level
overrides on top of project level overrides, and pass the resulted map to
KylinConfigExt.
This should work and code change is very minimal, straightforward.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673663#comment-15673663 ] kangkaisen commented on KYLIN-2180: --- OK，thank you. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15673645#comment-15673645 ] Shaofeng SHI commented on KYLIN-2180: - Hi Kaisen, this scenario is reasonable. I will look into it later (maybe someday in next week). Thank you. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672675#comment-15672675
]
kangkaisen commented on KYLIN-2180:
---
Hi, [~liyang.g...@gmail.com], [~Shaofengshi], please you review the
{{KylinCubeConfigExt}} when you have time, Thanks you.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662705#comment-15662705
]
kangkaisen commented on KYLIN-2180:
---
This is my solution:
Firstly, support to add project config in web UI, web API and project metadata.
I will make the project metadata is forward compatible.
Secondly, update the {{KylinConfigExt}} .
Finally, when use the {{cube.getConfig()}} to get the property, the property
will get from the config of cube, the config of project and the config of
server in turn.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
