[jira] [Created] (KYLIN-4168) Fix sonar reported static code issues phase 2
Yaqian Zhang created KYLIN-4168: --- Summary: Fix sonar reported static code issues phase 2 Key: KYLIN-4168 URL: https://issues.apache.org/jira/browse/KYLIN-4168 Project: Kylin Issue Type: Improvement Reporter: Yaqian Zhang -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] RupengWang commented on issue #792: KYLIN-4126 cube name validate code cause the wrong judge of streaming…
RupengWang commented on issue #792: KYLIN-4126 cube name validate code cause the wrong judge of streaming… URL: https://github.com/apache/kylin/pull/792#issuecomment-532027436 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4126) cube name validate code cause the wrong judge of streaming type
[ https://issues.apache.org/jira/browse/KYLIN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931012#comment-16931012 ] wangrupeng commented on KYLIN-4126: --- This issue fix the problem that kylin default properties cannot be loaded when create new cube or edit exist cube for streaming table. > cube name validate code cause the wrong judge of streaming type > > > Key: KYLIN-4126 > URL: https://issues.apache.org/jira/browse/KYLIN-4126 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: error.png, normal.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4126) cube name validate code cause the wrong judge of streaming type
[ https://issues.apache.org/jira/browse/KYLIN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangrupeng updated KYLIN-4126: -- Attachment: normal.png > cube name validate code cause the wrong judge of streaming type > > > Key: KYLIN-4126 > URL: https://issues.apache.org/jira/browse/KYLIN-4126 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: error.png, normal.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4126) cube name validate code cause the wrong judge of streaming type
[ https://issues.apache.org/jira/browse/KYLIN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangrupeng updated KYLIN-4126: -- Attachment: error.png > cube name validate code cause the wrong judge of streaming type > > > Key: KYLIN-4126 > URL: https://issues.apache.org/jira/browse/KYLIN-4126 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: error.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] coveralls commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive
coveralls commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839#issuecomment-531949334 ## Pull Request Test Coverage Report for [Build 5014](https://coveralls.io/builds/25731264) * **0** of **1** **(0.0%)** changed or added relevant line in **1** file are covered. * **2** unchanged lines in **1** file lost coverage. * Overall coverage increased (+**0.008%**) to **28.196%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [server-base/src/main/java/org/apache/kylin/rest/service/AccessService.java](https://coveralls.io/builds/25731264/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FAccessService.java#L425) | 0 | 1 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/25731264/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | Totals | [](https://coveralls.io/builds/25731264) | | :-- | --: | | Change from base [Build 5007](https://coveralls.io/builds/25692587): | 0.008% | | Covered Lines: | 23598 | | Relevant Lines: | 83692 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment
coveralls commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment URL: https://github.com/apache/kylin/pull/840#issuecomment-531917809 ## Pull Request Test Coverage Report for [Build 5015](https://coveralls.io/builds/25733744) * **0** of **23** **(0.0%)** changed or added relevant lines in **2** files are covered. * **7** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.178%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [stream-coordinator/src/main/java/org/apache/kylin/stream/coordinator/Coordinator.java](https://coveralls.io/builds/25733744/source?filename=stream-coordinator%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcoordinator%2FCoordinator.java#L1195) | 0 | 3 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/25733744/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L874) | 0 | 20 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/25733744/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/25733744/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/25733744) | | :-- | --: | | Change from base [Build 5007](https://coveralls.io/builds/25692587): | -0.01% | | Covered Lines: | 23589 | | Relevant Lines: | 83714 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4067) Speed up response of kylin cube page
[ https://issues.apache.org/jira/browse/KYLIN-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangrupeng updated KYLIN-4067: -- Attachment: duplicated_cube_name_request.png > Speed up response of kylin cube page > > > Key: KYLIN-4067 > URL: https://issues.apache.org/jira/browse/KYLIN-4067 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v3.0.0-alpha, v2.6.3 >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: duplicated_cube_name_request.png > > Original Estimate: 168h > Remaining Estimate: 168h > > Hi Team： > My Kylin project has more than 100 cubes. It is very slowly when open kylin > web page. > I find that kylin load all information of cubes when open cube page. It load > all information of all cubes at every cube in kylin project. The url of > request is "http://ip:port/kylin/api/cubes?limit=65535=0;. For > example, if one project has 10 cubes, this request will be called by 10 > times. But this information is only be used to determine whether the name is > duplicated when adding a new cube. > This page loading mechanism can be optimized. Getting all the information of > all cube only needs to be called when adding a new cube. > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (KYLIN-4126) cube name validate code cause the wrong judge of streaming type
[ https://issues.apache.org/jira/browse/KYLIN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangrupeng reassigned KYLIN-4126: - Assignee: luguosheng (was: wangrupeng) > cube name validate code cause the wrong judge of streaming type > > > Key: KYLIN-4126 > URL: https://issues.apache.org/jira/browse/KYLIN-4126 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (KYLIN-4126) cube name validate code cause the wrong judge of streaming type
[ https://issues.apache.org/jira/browse/KYLIN-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangrupeng reassigned KYLIN-4126: - Assignee: wangrupeng (was: luguosheng) > cube name validate code cause the wrong judge of streaming type > > > Key: KYLIN-4126 > URL: https://issues.apache.org/jira/browse/KYLIN-4126 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930485#comment-16930485 ] Yaqian Zhang commented on KYLIN-4149: - Verified. Test Evidence: 1、Load streaming table V2 called 'streaming_4149' and the topic of table receive message 100msg/s; !image-2019-09-16-20-07-11-847.png|width=421,height=141! 2、According to streaming table create model and cube called 'test_4149_cube'; !image-2019-09-16-19-49-20-648.png|width=357,height=104! 3、Modify Kafka Topic to 'kylin_streaming_topic' that is no message; !image-2019-09-16-20-04-51-179.png|width=449,height=142! 4、Clone ‘test_4149_cube’ to 'test_4149_cube_clone' and this cube will consume modified topic; !image-2019-09-16-20-09-08-218.png|width=391,height=136! 5、Modify Kafka cluster adress to another one adress have no Kafka and the cube will can't consume streaming data. !image-2019-09-16-20-02-07-687.png|width=433,height=84! > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png, > image-2019-09-16-20-02-07-687.png, image-2019-09-16-20-04-51-179.png, > image-2019-09-16-20-07-11-847.png, image-2019-09-16-20-09-08-218.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-20-09-08-218.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png, > image-2019-09-16-20-02-07-687.png, image-2019-09-16-20-04-51-179.png, > image-2019-09-16-20-07-11-847.png, image-2019-09-16-20-09-08-218.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-20-07-11-847.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png, > image-2019-09-16-20-02-07-687.png, image-2019-09-16-20-04-51-179.png, > image-2019-09-16-20-07-11-847.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-20-04-51-179.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png, > image-2019-09-16-20-02-07-687.png, image-2019-09-16-20-04-51-179.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-20-02-07-687.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png, > image-2019-09-16-20-02-07-687.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-19-57-45-117.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png, image-2019-09-16-19-57-45-117.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-19-51-12-214.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-19-51-36-338.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png, image-2019-09-16-19-51-12-214.png, > image-2019-09-16-19-51-36-338.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-19-49-20-648.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png, > image-2019-09-16-19-49-20-648.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (KYLIN-4121) Cleanup hive view intermediate tables after job be finished
[ https://issues.apache.org/jira/browse/KYLIN-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930447#comment-16930447 ] Yaqian Zhang edited comment on KYLIN-4121 at 9/16/19 11:45 AM: --- Verified. Test Evidence: 1、Use hive view table:fact table KYLIN_SALES_VIEW and lokkup table KYLIN_CAL_DT_VIEW create model and build cube.Two intermediate tables were generated during the construction process. !image-2019-09-16-19-18-59-663.png|width=483,height=460! 2、The intermediate table is deleted after the build is completed. !image-2019-09-16-19-26-31-446.png|width=434,height=579! was (Author: zhangyaqian): Verified. Test Evidence: 1、Use hive view table:fact table KYLIN_SALES_VIEW and lokkup table KYLIN_CAL_DT_VIEW create model and build cube.Two intermediate tables were generated during the construction process. !image-2019-09-16-19-18-59-663.png! 2、The intermediate table is deleted after the build is completed. !image-2019-09-16-19-26-31-446.png! > Cleanup hive view intermediate tables after job be finished > --- > > Key: KYLIN-4121 > URL: https://issues.apache.org/jira/browse/KYLIN-4121 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-18-59-663.png, > image-2019-09-16-19-26-31-446.png > > > Reported by community user: > I have a cube with a fact table join a lookup table in hive, and both are > hive view. I submit a job one time per hour. > > Kylin can drop the intermediate fact table , but doesn't drop the > intermediate lookup table . > > I check the source code , and find out that , at '13 step: Hive Cleanup' , > the source code has been commented. > > > It is a legacy issue. Now KYLIN-3515 has fixed that, so the cleanup can be > enabled. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4149) Allow user to edit streaming v2 table's kafka cluster address and topic name
[ https://issues.apache.org/jira/browse/KYLIN-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4149: Attachment: image-2019-09-16-19-44-24-499.png > Allow user to edit streaming v2 table's kafka cluster address and topic name > - > > Key: KYLIN-4149 > URL: https://issues.apache.org/jira/browse/KYLIN-4149 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-44-24-499.png > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KYLIN-4121) Cleanup hive view intermediate tables after job be finished
[ https://issues.apache.org/jira/browse/KYLIN-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930447#comment-16930447 ] Yaqian Zhang commented on KYLIN-4121: - Verified. Test Evidence: 1、Use hive view table:fact table KYLIN_SALES_VIEW and lokkup table KYLIN_CAL_DT_VIEW create model and build cube.Two intermediate tables were generated during the construction process. !image-2019-09-16-19-18-59-663.png! 2、The intermediate table is deleted after the build is completed. !image-2019-09-16-19-26-31-446.png! > Cleanup hive view intermediate tables after job be finished > --- > > Key: KYLIN-4121 > URL: https://issues.apache.org/jira/browse/KYLIN-4121 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-18-59-663.png, > image-2019-09-16-19-26-31-446.png > > > Reported by community user: > I have a cube with a fact table join a lookup table in hive, and both are > hive view. I submit a job one time per hour. > > Kylin can drop the intermediate fact table , but doesn't drop the > intermediate lookup table . > > I check the source code , and find out that , at '13 step: Hive Cleanup' , > the source code has been commented. > > > It is a legacy issue. Now KYLIN-3515 has fixed that, so the cleanup can be > enabled. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4121) Cleanup hive view intermediate tables after job be finished
[ https://issues.apache.org/jira/browse/KYLIN-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4121: Attachment: image-2019-09-16-19-26-31-446.png > Cleanup hive view intermediate tables after job be finished > --- > > Key: KYLIN-4121 > URL: https://issues.apache.org/jira/browse/KYLIN-4121 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-18-59-663.png, > image-2019-09-16-19-26-31-446.png > > > Reported by community user: > I have a cube with a fact table join a lookup table in hive, and both are > hive view. I submit a job one time per hour. > > Kylin can drop the intermediate fact table , but doesn't drop the > intermediate lookup table . > > I check the source code , and find out that , at '13 step: Hive Cleanup' , > the source code has been commented. > > > It is a legacy issue. Now KYLIN-3515 has fixed that, so the cleanup can be > enabled. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4121) Cleanup hive view intermediate tables after job be finished
[ https://issues.apache.org/jira/browse/KYLIN-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4121: Attachment: image-2019-09-16-19-18-59-663.png > Cleanup hive view intermediate tables after job be finished > --- > > Key: KYLIN-4121 > URL: https://issues.apache.org/jira/browse/KYLIN-4121 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta > > Attachments: image-2019-09-16-19-18-59-663.png > > > Reported by community user: > I have a cube with a fact table join a lookup table in hive, and both are > hive view. I submit a job one time per hour. > > Kylin can drop the intermediate fact table , but doesn't drop the > intermediate lookup table . > > I check the source code , and find out that , at '13 step: Hive Cleanup' , > the source code has been commented. > > > It is a legacy issue. Now KYLIN-3515 has fixed that, so the cleanup can be > enabled. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] codecov-io commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment
codecov-io commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment URL: https://github.com/apache/kylin/pull/840#issuecomment-531734445 # [Codecov](https://codecov.io/gh/apache/kylin/pull/840?src=pr=h1) Report > Merging [#840](https://codecov.io/gh/apache/kylin/pull/840?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/a2aca12ff3f8114be9c215ccee5e2ad74fe05f9d?src=pr=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/840?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#840 +/- ## === - Coverage 25.72% 25.7% -0.02% Complexity 61246124 === Files 14001400 Lines 83692 83714 +22 Branches 11732 11732 === - Hits 21526 21516 -10 - Misses60093 60128 +35 + Partials 20732070 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/840?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...g/apache/kylin/stream/coordinator/Coordinator.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-c3RyZWFtLWNvb3JkaW5hdG9yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdHJlYW0vY29vcmRpbmF0b3IvQ29vcmRpbmF0b3IuamF2YQ==) | `24.82% <0%> (-0.06%)` | `35 <0> (ø)` | | | [...g/apache/kylin/rest/controller/CubeController.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9DdWJlQ29udHJvbGxlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ream/core/storage/columnar/ColumnarStoreCache.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTdG9yZUNhY2hlLmphdmE=) | `51.16% <0%> (-3.49%)` | `8% <0%> (ø)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...eam/core/storage/columnar/DataSegmentFragment.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvRGF0YVNlZ21lbnRGcmFnbWVudC5qYXZh) | `57.37% <0%> (-1.64%)` | `17% <0%> (-1%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/840/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/840?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/840?src=pr=footer). Last update [a2aca12...846282d](https://codecov.io/gh/apache/kylin/pull/840?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4162) After drop the build task on the monitor page, subsequent segments cannot be constructed.
[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930423#comment-16930423 ] ASF GitHub Bot commented on KYLIN-4162: --- zhangayqian commented on pull request #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment URL: https://github.com/apache/kylin/pull/840 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > - > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] zhangayqian opened a new pull request #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment
zhangayqian opened a new pull request #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment URL: https://github.com/apache/kylin/pull/840 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asf-ci commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment
asf-ci commented on issue #840: KYLIN-4162 After the paragraph is dropped, continue to build the subsequent segment and provide REST API to resubmit the segment URL: https://github.com/apache/kylin/pull/840#issuecomment-531727012 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-4166) kylin parse sql error
[
https://issues.apache.org/jira/browse/KYLIN-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
phil.zhang updated KYLIN-4166:
--
Description:
{quote} 1. I use tableau on kylin, tableau has generate a sql :
'''
SELECT "X___SQL___"."FLIGHT_TYPE" AS "FLIGHT_TYPE",
SUM("X___SQL___"."SHOPPING_SUCCESS_COUNT") AS "sum_SHOPPING_SUCCESS_COUNT_ok"
FROM ( select *, case when cast(DURATION_LEVEL as varchar)
like '1' then '>=0s,<3s' when cast(DURATION_LEVEL as varchar) like '2'
then '>=3s,<4s' when cast(DURATION_LEVEL as varchar) like '3' then
'>=4s,<5s' when cast(DURATION_LEVEL as varchar) like '4' then
*'>=5s,<6s'* when cast(DURATION_LEVEL as varchar) like '5' then
'>=6s,<7s' when cast(DURATION_LEVEL as varchar) like '6' then
'>=7s,<8s' when cast(DURATION_LEVEL as varchar) like '7' then
'>=8s,<9s' when cast(DURATION_LEVEL as varchar) like '8' then
'>=9s,<10s' when cast(DURATION_LEVEL as varchar) like '9' then
'>=10s,<12s' when cast(DURATION_LEVEL as varchar) like '10' then
'>=12s,<14s' when cast(DURATION_LEVEL as varchar) like '11' then
'>=14s,<16s' when cast(DURATION_LEVEL as varchar) like '12' then
'>=16s,<18s' when cast(DURATION_LEVEL as varchar) like '13' then
'>=18s,<20s' when cast(DURATION_LEVEL as varchar) like '14' then
'>=20s,<30s' when cast(DURATION_LEVEL as varchar) like '15' then
'>=30s,<40s' when cast(DURATION_LEVEL as varchar) like '16' then
'>=40s,<50s' when cast(DURATION_LEVEL as varchar) like '17' then
'>=50s' else 'null' end as DURATION_LEVEL_TRANS from
DM_AIR_API_DURATION_SPB_D ) "X___SQL___" WHERE
("X___SQL___"."DURATION_LEVEL_TRANS" = *'>=5s,<6s'*) GROUP BY 1
'''
it seems that the sql is correct, but the result is unexpect, it returns
null .{quote}
!http://mail.163.com/js6/s?func=mbox:getMessageData=270:xtbBDhAtQVrbEIH+vQAAsh=4!
{quote} 2. But actually, it should returns some rows because there exist data
fit the query, let's change the sql form to check it.
'''
SELECT "FLIGHT_TYPE" AS "FLIGHT_TYPE", SUM("SHOPPING_SUCCESS_COUNT") AS
"sum_SHOPPING_SUCCESS_COUNT_ok" from DM_AIR_API_DURATION_SPB_D WHERE
("DURATION_LEVEL" = 4 and "DAY_NUM"='2019-09-10') GROUP BY 1
'''{quote}
!http://mail.163.com/js6/s?func=mbox:getMessageData=270:xtbBDhAtQVrbEIH+vQAAsh=4!
{quote}and when i write sql like this
'''
SELECT flight_type,SUM("SHOPPING_SUCCESS_COUNT") AS
"sum_SHOPPING_SUCCESS_COUNT_ok" from DM_AIR_API_DURATION_SPB_D group by
flight_type
'''
i got result {color:#33}like this{color}{quote}
{color:#33}!http://mail.163.com/js6/s?func=mbox:getMessageData=270:xtbBDhAtQVrbEIH+vQAAsh=5!{color}
{quote}it is same to the result of
'''
SELECT "X___SQL___"."FLIGHT_TYPE" AS "FLIGHT_TYPE",
SUM("X___SQL___"."SHOPPING_SUCCESS_COUNT") AS "sum_SHOPPING_SUCCESS_COUNT_ok"
FROM ( select *, case when cast(DURATION_LEVEL as varchar)
like '1' then *'>=0s,<3s'* when cast(DURATION_LEVEL as varchar) like
'2' then '>=3s,<4s' when cast(DURATION_LEVEL as varchar) like '3' then
'>=4s,<5s' when cast(DURATION_LEVEL as varchar) like '4' then
'>=5s,<6s' when cast(DURATION_LEVEL as varchar) like '5' then
'>=6s,<7s' when cast(DURATION_LEVEL as varchar) like '6' then
'>=7s,<8s' when cast(DURATION_LEVEL as varchar) like '7' then
'>=8s,<9s' when cast(DURATION_LEVEL as varchar) like '8' then
'>=9s,<10s' when cast(DURATION_LEVEL as varchar) like '9' then
'>=10s,<12s' when cast(DURATION_LEVEL as varchar) like '10' then
'>=12s,<14s' when cast(DURATION_LEVEL as varchar) like '11' then
'>=14s,<16s' when cast(DURATION_LEVEL as varchar) like '12' then
'>=16s,<18s' when cast(DURATION_LEVEL as varchar) like '13' then
'>=18s,<20s' when cast(DURATION_LEVEL as varchar) like '14' then
'>=20s,<30s' when cast(DURATION_LEVEL as varchar) like '15' then
'>=30s,<40s' when cast(DURATION_LEVEL as varchar) like '16' then
'>=40s,<50s' when cast(DURATION_LEVEL as varchar) like '17' then
'>=50s' else 'null' end as DURATION_LEVEL_TRANS from
DM_AIR_API_DURATION_SPB_D ) "X___SQL___" WHERE
("X___SQL___"."DURATION_LEVEL_TRANS" = *'>=0s,<3s'*) GROUP BY 1
'''{quote}
!http://mail.163.com/js6/s?func=mbox:getMessageData=270:xtbBDhAtQVrbEIH+vQAAsh=6!
{quote} *I guess maybe there are some bugs about parsing sql like upper ? I
hope someone can help me to deal this problem.*{quote}
was:
我在kylin上使用tableau，tableau生成了一个sql：
"""
选择"X ___ SQL ___"。"FLIGHT_TYPE"AS"FLIGHT_TYPE"，SUM（"X ___ SQL
___"。"SHOPPING_SUCCESS_COUNT"）AS"sum_SHOPPING_SUCCESS_COUNT_ok"FROM（选择*，转换时的情况（DURATION_LEVEL
as varchar），如'1'，然后'> = 0s，当施放时（3）'（'D'为'var'），然后'> =
3s，<4s'，当施放时（DURATION_LEVEL作为varchar）像'3'然后'> =
[GitHub] [kylin] codecov-io commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive
codecov-io commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839#issuecomment-531692881 # [Codecov](https://codecov.io/gh/apache/kylin/pull/839?src=pr=h1) Report > Merging [#839](https://codecov.io/gh/apache/kylin/pull/839?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/a2aca12ff3f8114be9c215ccee5e2ad74fe05f9d?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/839?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #839 +/- ## - Coverage 25.72% 25.71% -0.01% Complexity 6124 6124 Files 1400 1400 Lines 8369283692 Branches 1173211732 - Hits 2152621521 -5 - Misses6009360097 +4 - Partials 2073 2074 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/839?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...a/org/apache/kylin/rest/service/AccessService.java](https://codecov.io/gh/apache/kylin/pull/839/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9BY2Nlc3NTZXJ2aWNlLmphdmE=) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/839/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/839/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/839?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/839?src=pr=footer). Last update [a2aca12...a9c4571](https://codecov.io/gh/apache/kylin/pull/839?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930322#comment-16930322 ] ASF GitHub Bot commented on KYLIN-4103: --- lshmouse commented on pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839 See: https://jira.apache.org/jira/browse/KYLIN-4103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [kylin] asf-ci commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive
asf-ci commented on issue #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839#issuecomment-531684734 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] lshmouse opened a new pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive
lshmouse opened a new pull request #839: KYLIN-4103: Make the user string in granting operation of project is case insensitive URL: https://github.com/apache/kylin/pull/839 See: https://jira.apache.org/jira/browse/KYLIN-4103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4162) After drop the build task on the monitor page, subsequent segments cannot be constructed.
[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930293#comment-16930293 ] Xiaoxiang Yu commented on KYLIN-4162: - It seems it duplicated with https://issues.apache.org/jira/browse/KYLIN-4135. > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > - > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4153) Failed to read big resource /dict/xxxx at "Build Dimension Dictionary" Step
[
https://issues.apache.org/jira/browse/KYLIN-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4153:
Fix Version/s: v2.6.4
v3.0.0-beta
> Failed to read big resource /dict/ at "Build Dimension Dictionary" Step
>
>
> Key: KYLIN-4153
> URL: https://issues.apache.org/jira/browse/KYLIN-4153
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.0
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> At the version of *Kylin 2.6.0*, kylin team has introduce an important
> refactor of Kylin's Metadata Store, which add a lot of enhancement such as
> upload/download metadata concurrently, store metadata with JDBC etc. Please
> refer to https://issues.apache.org/jira/browse/KYLIN-3671 for detail.
>
> When kylin want to save a *big resource*(such as dict or snapshot) into
> metadata store, it won't store it into metadata store(HBase or RDBMS)
> directly. Instead, kylin will first {color:red}save it into HDFS(Step
> 1){color}, and then {color:red}write a empty byte array as marker into
> metadata store(Step 2) {color}. If first action succeed and second action
> failed, a rollback method will be called to revert modification for HDFS
> files. We could regard it as a complete and atomic transaction.
>
> {color:#0747A6}Here is part of the source code added in KYLIN-3671.{color}
> Check it at
> https://github.com/apache/kylin/blob/8737bc1f555a2789a67462c8f8420b6ab3be97ce/core-common/src/main/java/org/apache/kylin/common/persistence/PushdownResourceStore.java#L58
> .
> {code:java}
> final void putBigResource(String resPath, ContentWriter content, long newTS)
> throws IOException {
> // pushdown the big resource to DFS file
> RollbackablePushdown pushdown = writePushdown(resPath, content); // Step
> 1: write big resource into HDFS
> try {
> // write a marker in resource store, to indicate the resource is now
> available
> logger.debug("Writing marker for big resource {}", resPath);
> putResourceWithRetry(resPath,
> ContentWriter.create(BytesUtil.EMPTY_BYTE_ARRAY), newTS); // Step 2: write
> marker into HBase/RDBMS
> } catch (Throwable ex) {
> pushdown.rollback();
> throw ex;
> } finally {
> pushdown.close();
> }
> }
> {code}
>
>
>
> But in some case, both step 1 and step 2 succeed but an exception still
> throwed in step 2,{color:red} the rollback won't clear marker written in Step
> 2{color}, which break the atomicity of this put action, thus cause the
> FileNotFoundException when Kylin want to read that dict later.
>
>
>
> {color:#0747A6}Here is part of reporter's kylin.log of incomplete rollback
> action.{color}
>
>
> {noformat}
> 2019-08-29 05:13:51,237 INFO [Scheduler 169045403 Job
> ca4a4a08-54e2-b922-70bb-2aa2bf58709f-492] dict.DictionaryManager:388 : Saving
> dictionary at
> /dict/KYLIN_VIEW.USER_SECRET_TABLE/COUNTRY/66292068-e8eb-975a-3e44-b56c933c14cc.dict
> 2019-08-29 05:13:51,238 DEBUG [Scheduler 169045403 Job
> ca4a4a08-54e2-b922-70bb-2aa2bf58709f-492] persistence.HDFSResourceStore:98 :
> Writing pushdown file
> /kylin/kylin_metadata/resources/dict/KYLIN_VIEW.USER_SECRET_TABLE/COUNTRY/66292068-e8eb-975a-3e44-b56c933c14cc.dict.temp.-1798610090
> 2019-08-29 05:13:51,256 DEBUG [Scheduler 169045403 Job
> ca4a4a08-54e2-b922-70bb-2aa2bf58709f-492] persistence.HDFSResourceStore:117 :
> Move
> /kylin/kylin_metadata/resources/dict/KYLIN_VIEW.USER_SECRET_TABLE/COUNTRY/66292068-e8eb-975a-3e44-b56c933c14cc.dict.temp.-1798610090
> to
> /kylin/kylin_metadata/resources/dict/KYLIN_VIEW.USER_SECRET_TABLE/COUNTRY/66292068-e8eb-975a-3e44-b56c933c14cc.dict
> 2019-08-29 05:13:51,258 DEBUG [Scheduler 169045403 Job
> ca4a4a08-54e2-b922-70bb-2aa2bf58709f-492] persistence.HDFSResourceStore:65 :
> Writing marker for big resource
> /dict/KYLIN_VIEW.USER_SECRET_TABLE/COUNTRY/66292068-e8eb-975a-3e44-b56c933c14cc.dict
> 2019-08-29 05:13:56,263 WARN
> [hconnection-0x56f3258e-shared--pool10944-t54867] client.AsyncProcess:1263 :
> #10545, table=kylin_metadata, attempt=1/1 failed=1ops, last exception:
> java.io.IOException: Call to tx-dn41.data/10.14.243.51:60020 failed on local
> exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=2662317,
> waitTime=5001, operationTimeout=5000 expired. on
> tx-dn41.data,60020,1565943919204, tracking started Thu Aug 29 05:13:51
> GMT+08:00 2019; not retrying 1 - final failure
> 2019-08-29 05:13:56,266 ERROR [Scheduler 169045403 Job
> ca4a4a08-54e2-b922-70bb-2aa2bf58709f-492] persistence.HDFSResourceStore:134 :
> Rollback
>
[jira] [Updated] (KYLIN-4162) After drop the build task on the monitor page, subsequent segments cannot be constructed.
[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-4162: Fix Version/s: (was: Future) v3.0.0-beta > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > - > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4010) TimeZone is hard-coded in function makeSegmentName for class CubeSegment
[ https://issues.apache.org/jira/browse/KYLIN-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-4010: Fix Version/s: v3.0.0-beta > TimeZone is hard-coded in function makeSegmentName for class CubeSegment > > > Key: KYLIN-4010 > URL: https://issues.apache.org/jira/browse/KYLIN-4010 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v3.0.0-alpha >Reporter: zengrui >Assignee: Xiaoxiang Yu >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: image-2019-07-15-17-15-31-209.png, > image-2019-07-15-17-17-04-029.png, image-2019-07-15-17-17-39-568.png > > > In Real-Time Streaming Cube when I send some records to kafka topic, the > tmestamp for the record is 2019-01-01 00:00:00.000, but kylin create a > segment named 2018123116_2018123117. > Then I found that TimeZone is hard-coded to "GMT" in function makeSegmentName > for class CubeSegment. I think that it should be config in kylin.properties. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (KYLIN-4141) Build Global Dictionary in no time
[ https://issues.apache.org/jira/browse/KYLIN-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-4141: Fix Version/s: v3.0.0-beta > Build Global Dictionary in no time > -- > > Key: KYLIN-4141 > URL: https://issues.apache.org/jira/browse/KYLIN-4141 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-beta >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
