[jira] [Created] (KYLIN-3456) Cube level's snapshot config does not work
Xinbei Fu created KYLIN-3456: Summary: Cube level's snapshot config does not work Key: KYLIN-3456 URL: https://issues.apache.org/jira/browse/KYLIN-3456 Project: Kylin Issue Type: Bug Reporter: Xinbei Fu Configuration kylin.snapshot.max-mb does not work when set in cubes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3452) Optimize spark cubing memory footprint
[ https://issues.apache.org/jira/browse/KYLIN-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3452: --- Assignee: Shaofeng SHI > Optimize spark cubing memory footprint > -- > > Key: KYLIN-3452 > URL: https://issues.apache.org/jira/browse/KYLIN-3452 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Noticed that there are some steps can be optimized: > 1) Cuboid.findForMandatory() will create new object each time, which this can > be cached; > 2) Executor repeatedly load dictionary, not hit the cache; > 3) NDCuboidBuilder and RowkeyEncoder, can optimize for less array copy; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3452) Optimize spark cubing memory footprint
[ https://issues.apache.org/jira/browse/KYLIN-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3452: Fix Version/s: v2.5.0 > Optimize spark cubing memory footprint > -- > > Key: KYLIN-3452 > URL: https://issues.apache.org/jira/browse/KYLIN-3452 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Noticed that there are some steps can be optimized: > 1) Cuboid.findForMandatory() will create new object each time, which this can > be cached; > 2) Executor repeatedly load dictionary, not hit the cache; > 3) NDCuboidBuilder and RowkeyEncoder, can optimize for less array copy; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-3454: --- Assignee: Xinbei Fu > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Assignee: Xinbei Fu >Priority: Major > Fix For: v2.5.0 > > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck if multiple > threads are trying to modify this variable. > !Screen Shot 2018-07-16 at 2.36.39 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-2933) Fix compilation against the Kafka 1.0.0 release
[
https://issues.apache.org/jira/browse/KYLIN-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2933:
---
Assignee: 翟娜 (was: Shaofeng SHI)
> Fix compilation against the Kafka 1.0.0 release
> ---
>
> Key: KYLIN-2933
> URL: https://issues.apache.org/jira/browse/KYLIN-2933
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: 翟娜
>Priority: Major
> Fix For: v2.5.0
>
>
> Due to change of class location of SecurityProtocol, KafkaClusterConfig.java
> needs to be modified so that it compiles against the Kafka 1.0.0 release:
> {code}
> -import org.apache.kafka.common.protocol.SecurityProtocol;
> +import org.apache.kafka.common.security.auth.SecurityProtocol;
> {code}
> A compatibility layer may be needed if we support multiple Kafka releases
> simultaneously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3455) Collect cpu time for coprocessor thread
Zhong Yanghong created KYLIN-3455: - Summary: Collect cpu time for coprocessor thread Key: KYLIN-3455 URL: https://issues.apache.org/jira/browse/KYLIN-3455 Project: Kylin Issue Type: Improvement Reporter: Zhong Yanghong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3454: Fix Version/s: v2.5.0 > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Fix For: v2.5.0 > > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck if multiple > threads are trying to modify this variable. > !Screen Shot 2018-07-16 at 2.36.39 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3453) Improve cube size estimation for TOPN, COUNT DISTINCT
[ https://issues.apache.org/jira/browse/KYLIN-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3453: Fix Version/s: v2.5.0 > Improve cube size estimation for TOPN, COUNT DISTINCT > - > > Key: KYLIN-3453 > URL: https://issues.apache.org/jira/browse/KYLIN-3453 > Project: Kylin > Issue Type: Improvement >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.5.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3168) CubeHFileJob should use currentHBaseConfiguration but not new create hbase configuration
[ https://issues.apache.org/jira/browse/KYLIN-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyingjun resolved KYLIN-3168. -- Resolution: Fixed > CubeHFileJob should use currentHBaseConfiguration but not new create hbase > configuration > > > Key: KYLIN-3168 > URL: https://issues.apache.org/jira/browse/KYLIN-3168 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.2.0, v2.3.0 >Reporter: wuyingjun >Assignee: wuyingjun >Priority: Major > Fix For: v2.4.0 > > Attachments: AfterModified.png, CubeHFileJob_Exception.png, > KYLIN-3168.patch, KYLIN-3438.patch, Scheduler.png, job_queue.png, > queue_set.png, result.png, 飞信截图20180124232443.png > > > CubeHFileJob use new hbase configuration is not correct because when the > zookeeper quorum is not localhost (job worker)， kylin may encounter zookeeper > ConnectionLossException. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3438) mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step
[ https://issues.apache.org/jira/browse/KYLIN-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyingjun resolved KYLIN-3438. -- Resolution: Fixed > mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step > > > Key: KYLIN-3438 > URL: https://issues.apache.org/jira/browse/KYLIN-3438 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.0 >Reporter: powerinf >Assignee: wuyingjun >Priority: Major > Fix For: v2.4.1, v2.5.0 > > Attachments: KYLIN-3438.patch, Scheduler.png, job_queue.png, > queue_set.png, result.png > > > set the mapreduce.job.queuename does not work until the ' Convert Cuboid Data > to HFile' step ，but precious steps can work well。 > I put the mapreduce.job.queuename property in all conf file including > kylin_hive_conf.xml、kylin_job_conf_inmem.xml、kylin_job_conf.xml ，but does not > work，it alse to use the default queue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3438) mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step
[ https://issues.apache.org/jira/browse/KYLIN-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545295#comment-16545295 ] ASF subversion and git services commented on KYLIN-3438: Commit d5affd97fd95b6e915b97502eb5639153a3d6365 in kylin's branch refs/heads/master from wyj [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d5affd9 ] KYLIN-3438 mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step Signed-off-by: shaofengshi > mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step > > > Key: KYLIN-3438 > URL: https://issues.apache.org/jira/browse/KYLIN-3438 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.0 >Reporter: powerinf >Assignee: wuyingjun >Priority: Major > Fix For: v2.4.1, v2.5.0 > > Attachments: KYLIN-3438.patch, Scheduler.png, job_queue.png, > queue_set.png, result.png > > > set the mapreduce.job.queuename does not work until the ' Convert Cuboid Data > to HFile' step ，but precious steps can work well。 > I put the mapreduce.job.queuename property in all conf file including > kylin_hive_conf.xml、kylin_job_conf_inmem.xml、kylin_job_conf.xml ，but does not > work，it alse to use the default queue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3438) mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step
[ https://issues.apache.org/jira/browse/KYLIN-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545264#comment-16545264 ] ASF subversion and git services commented on KYLIN-3438: Commit 273f7cc2990a48ccc888029635a90a30a095062b in kylin's branch refs/heads/2.4.x from wyj [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=273f7cc ] KYLIN-3438 mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step Signed-off-by: shaofengshi > mapreduce.job.queuename does not work at 'Convert Cuboid Data to HFile' Step > > > Key: KYLIN-3438 > URL: https://issues.apache.org/jira/browse/KYLIN-3438 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.0 >Reporter: powerinf >Assignee: wuyingjun >Priority: Major > Fix For: v2.4.1, v2.5.0 > > Attachments: KYLIN-3438.patch, Scheduler.png, job_queue.png, > queue_set.png, result.png > > > set the mapreduce.job.queuename does not work until the ' Convert Cuboid Data > to HFile' step ，but precious steps can work well。 > I put the mapreduce.job.queuename property in all conf file including > kylin_hive_conf.xml、kylin_job_conf_inmem.xml、kylin_job_conf.xml ，but does not > work，it alse to use the default queue -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2312) Display Server Config/Environment by order in system tab
[ https://issues.apache.org/jira/browse/KYLIN-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544969#comment-16544969 ] hongbin ma commented on KYLIN-2312: --- after KYLIN-2659 this is no longer working, and we haven't received too many complains afterwards, so maybe the requirement here is not very strong > Display Server Config/Environment by order in system tab > - > > Key: KYLIN-2312 > URL: https://issues.apache.org/jira/browse/KYLIN-2312 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > The system tab page shows Server Config and Environment, it's useful for > debugging, but the item order is undetermined currently. The Config should > show the same order as the properties file. The Environment should show the > items order by name. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Description: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck if multiple threads are trying to modify this variable. !Screen Shot 2018-07-16 at 2.36.39 PM.png! was: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !Screen Shot 2018-07-16 at 2.36.39 PM.png! > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck if multiple > threads are trying to modify this variable. > !Screen Shot 2018-07-16 at 2.36.39 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Description: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !Screen Shot 2018-07-16 at 2.36.39 PM.png! was: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck > !Screen Shot 2018-07-16 at 2.36.39 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Attachment: Screen Shot 2018-07-16 at 2.36.39 PM.png > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Description: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck was: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !Screen Shot 2018-07-16 at 2.36.39 PM.png! > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Attachment: (was: Screen Shot 2018-07-16 at 2.36.39 PM.png) > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Attachment: Screen Shot 2018-07-16 at 2.36.39 PM.png > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck > !https://user-images.githubusercontent.com/15867197/41578546-751fe224-73c5-11e8-9f50-e98d67f26160.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
[ https://issues.apache.org/jira/browse/KYLIN-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinbei Fu updated KYLIN-3454: - Description: Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !Screen Shot 2018-07-16 at 2.36.39 PM.png! was:Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !https://user-images.githubusercontent.com/15867197/41578546-751fe224-73c5-11e8-9f50-e98d67f26160.png! > Fix potential thread-safe problem in ResourceTool > - > > Key: KYLIN-3454 > URL: https://issues.apache.org/jira/browse/KYLIN-3454 > Project: Kylin > Issue Type: Improvement >Reporter: Xinbei Fu >Priority: Major > Attachments: Screen Shot 2018-07-16 at 2.36.39 PM.png > > > Class org.apache.kylin.common.persistence.ResourceTool are called by other > methods, not only by command lines, so it got this potential thread-safe > problem regarding the static variable pathsSkipChildrenCheck > !Screen Shot 2018-07-16 at 2.36.39 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3454) Fix potential thread-safe problem in ResourceTool
Xinbei Fu created KYLIN-3454: Summary: Fix potential thread-safe problem in ResourceTool Key: KYLIN-3454 URL: https://issues.apache.org/jira/browse/KYLIN-3454 Project: Kylin Issue Type: Improvement Reporter: Xinbei Fu Class org.apache.kylin.common.persistence.ResourceTool are called by other methods, not only by command lines, so it got this potential thread-safe problem regarding the static variable pathsSkipChildrenCheck !https://user-images.githubusercontent.com/15867197/41578546-751fe224-73c5-11e8-9f50-e98d67f26160.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
