[jira] [Commented] (KYLIN-2720) Should not allow user to access to all tables' metadata of a project
[ https://issues.apache.org/jira/browse/KYLIN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150072#comment-16150072 ] Shaofeng SHI commented on KYLIN-2720: - Hi mingming, yes I see your patch is very clear. Since 2.1, the design is changed, and query pushdown is introduced, that means without cube user can query the tables (you can take a try). Then filtering the tables by cube is not needed any more. [~mahongbin] can correct me if my understanding is wrong. > Should not allow user to access to all tables' metadata of a project > > > Key: KYLIN-2720 > URL: https://issues.apache.org/jira/browse/KYLIN-2720 > Project: Kylin > Issue Type: Improvement >Reporter: qiumingming >Assignee: qiumingming > Attachments: KYLIN-2720.patch > > > Currently, user can access to all tables and columns metadata of a specific > project as long as he can access to this project, which is not reasonable. > User should just allow to access to tables that he owned cubes dependent to. > However, user can see some other tables in the web UI in current version. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2827) cube已经建好了，再insight 查询回报错
[ https://issues.apache.org/jira/browse/KYLIN-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150069#comment-16150069 ] Shaofeng SHI commented on KYLIN-2827: - Hi Liuxd, could you drop the question to apache kylin dev mailing list (d...@kylin.apache.org)? You need subscribe it (send an email to dev-subscr...@kylin.apache.org) if you haven't. JIRA is for tracking bug and requirements, not for problem solving. You issue seems be an environment issue. Thanks for the coorporation. > cube已经建好了，再insight 查询回报错 > - > > Key: KYLIN-2827 > URL: https://issues.apache.org/jira/browse/KYLIN-2827 > Project: Kylin > Issue Type: Bug >Reporter: liuxd > > Error while executing SQL "select * from GDL_LOANRPY_DET LIMIT 5": > org.apache.hadoop.hbase.exceptions.UnknownProtocolException: > org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered > coprocessor service found for name CubeVisitService in region > KYLIN_ZW5LTEKVAL,,1504166387142.432256ac271b98fc6afd29bc7e054502. at > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7582) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1890) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1872) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117) at > org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104) at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at > java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2831) Does kylin do not support underlined fields?
dengpitt created KYLIN-2831: --- Summary: Does kylin do not support underlined fields? Key: KYLIN-2831 URL: https://issues.apache.org/jira/browse/KYLIN-2831 Project: Kylin Issue Type: Improvement Components: General Affects Versions: v2.0.0 Reporter: dengpitt Fix For: Future When I choose the dimension contains _uid will report the following error: 17/08/31 10:43:35 [main]: INFO parse.ParseDriver: Parsing command: INSERT OVERWRITE TABLE kylin_intermediate_uc_3cf963bd_53ca_47d6_ae0c_a71b364fc558 SELECT USER_GAMBLING._UID ,USER_GAMBLING.BID FROM DEFAULT.USER_GAMBLING as USER_GAMBLING WHERE (USER_GAMBLING.TM >= 20170831 AND USER_GAMBLING.TM < 20170831) NoViableAltException(13@[127:1: selectItem : ( ( tableAllColumns )=> tableAllColumns -> ^( TOK_SELEXPR tableAllColumns ) | ( expression ( ( ( KW_AS )? identifier ) | ( KW_AS LPAREN identifier ( COMMA identifier )* RPAREN ) )? ) -> ^( TOK_SELEXPR expression ( identifier )* ) );]) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser$DFA17.specialStateTransition(HiveParser_SelectClauseParser.java:11835) at org.antlr.runtime.DFA.predict(DFA.java:80) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectItem(HiveParser_SelectClauseParser.java:2853) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1373) at org.apache.hadoop.hive.ql.parse.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParser.java:1128) at org.apache.hadoop.hive.ql.parse.HiveParser.selectClause(HiveParser.java:45827) at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:41495) at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:41230) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:40413) at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:40283) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1590) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1109) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:202) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:396) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) FAILED: ParseException line 3:14 cannot recognize input near 'USER_GAMBLING' '.' '_UID' in selection target 17/08/31 10:43:35 [main]: ERROR ql.Driver: FAILED: ParseException line 3:14 cannot recognize input near 'USER_GAMBLING' '.' '_UID' in selection target org.apache.hadoop.hive.ql.parse.ParseException: line 3:14 cannot recognize input near 'USER_GAMBLING' '.' '_UID' in selection target at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:205) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:396) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(Cli
[jira] [Updated] (KYLIN-2802) Enable cube planner phase one
[ https://issues.apache.org/jira/browse/KYLIN-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2802: -- Description: This phase happens during cube building. To be able to trigger cube planner, one of the following preconditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. We also introduce a threshold. If the total number of defined cuboids is less than this threshold, cube planner should not be triggered. was: This phase happens during cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. We also introduce a threshold. If the total number of defined cuboids is less than this threshold, cube planner should not be triggered. > Enable cube planner phase one > - > > Key: KYLIN-2802 > URL: https://issues.apache.org/jira/browse/KYLIN-2802 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > This phase happens during cube building. To be able to trigger cube planner, > one of the following preconditions should be satisfied. > * The cube is empty > * Job is for refreshing the only one segment. > Since cube planner is based on cuboid row count, if the first segment is > empty and there's no useful row count info, cube planner should not be > triggered in this phase. > We also introduce a threshold. If the total number of defined cuboids is less > than this threshold, cube planner should not be triggered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2802) Enable cube planner phase one
[ https://issues.apache.org/jira/browse/KYLIN-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2802: -- Description: This phase happens during cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. We also introduce a threshold. If the total number of defined cuboids is less than this threshold, cube planner should not be triggered. was: This phase happens at cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. We also introduce a threshold. If the total number of defined cuboids is less than this threshold, cube planner should not be triggered. > Enable cube planner phase one > - > > Key: KYLIN-2802 > URL: https://issues.apache.org/jira/browse/KYLIN-2802 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > This phase happens during cube building. To trigger cube planner, one of the > following conditions should be satisfied. > * The cube is empty > * Job is for refreshing the only one segment. > Since cube planner is based on cuboid row count, if the first segment is > empty and there's no useful row count info, cube planner should not be > triggered in this phase. > We also introduce a threshold. If the total number of defined cuboids is less > than this threshold, cube planner should not be triggered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2802) Enable cube planner phase one
[ https://issues.apache.org/jira/browse/KYLIN-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2802: -- Description: This phase happens at cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. We also introduce a threshold. If the total number of defined cuboids is less than this threshold, cube planner should not be triggered. was: This phase happens at cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. > Enable cube planner phase one > - > > Key: KYLIN-2802 > URL: https://issues.apache.org/jira/browse/KYLIN-2802 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > This phase happens at cube building. To trigger cube planner, one of the > following conditions should be satisfied. > * The cube is empty > * Job is for refreshing the only one segment. > Since cube planner is based on cuboid row count, if the first segment is > empty and there's no useful row count info, cube planner should not be > triggered in this phase. > We also introduce a threshold. If the total number of defined cuboids is less > than this threshold, cube planner should not be triggered. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2734: -- Description: The motivation of this feature is to provide users a way to directly designate a dimension set to be prebuilt. It's an additional way to define prebuilt cuboids besides static rules and will only be feasible for an empty cube. If hot cuboids are imported during cube design phase and it is able to go through cube planner algorithm to get recommended cuboids, regardless whether the total number of defined cuboids is less than the threshold to trigger cube planner phase one, the cube planner phase one will be triggered and these imported cuboids will be regarded as mandatory cuboids for cube planner algorithm. Corresponding to the import feature, hot cuboids should be able to exported for future cube upgrading and so on. was: The motivation of this feature is to provide users a way to directly designate a dimension set to be prebuilt. It's an additional way to define prebuilt cuboids besides static rules and will only be feasible for an empty cube. If hot cuboids are imported during cube design phase, cube planner phase one will be triggered and these imported cuboids will be regarded as mandatory cuboids for cube planner algorithm. Corresponding to the import feature, hot cuboids should be able to exported for future cube upgrading and so on. > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase and it is able to > go through cube planner algorithm to get recommended cuboids, regardless > whether the total number of defined cuboids is less than the threshold to > trigger cube planner phase one, the cube planner phase one will be triggered > and these imported cuboids will be regarded as mandatory cuboids for cube > planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2734: -- Description: The motivation of this feature is to provide users a way to directly designate a dimension set to be prebuilt. It's an additional way to define prebuilt cuboids besides static rules and will only be feasible for an empty cube. If hot cuboids are imported during cube design phase, cube planner phase one will be triggered and these imported cuboids will be regarded as mandatory cuboids for cube planner algorithm. Corresponding to the import feature, hot cuboids should be able to exported for future cube upgrading and so on. > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > The motivation of this feature is to provide users a way to directly > designate a dimension set to be prebuilt. It's an additional way to define > prebuilt cuboids besides static rules and will only be feasible for an empty > cube. If hot cuboids are imported during cube design phase, cube planner > phase one will be triggered and these imported cuboids will be regarded as > mandatory cuboids for cube planner algorithm. > Corresponding to the import feature, hot cuboids should be able to exported > for future cube upgrading and so on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2822: -- Description: To visualize cuboid tree, we introduce sunburst chart. At this chart, the base cuboid is at the center and the parent-child relationship is intuitive. What's more, additional info can be displayed for each cuboid, like cuboid (target) query count, cuboid row count, etc. > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > To visualize cuboid tree, we introduce sunburst chart. At this chart, the > base cuboid is at the center and the parent-child relationship is intuitive. > What's more, additional info can be displayed for each cuboid, like cuboid > (target) query count, cuboid row count, etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2733: -- Description: Optimize job is for adjusting existing cuboid set. Its data source is base cuboid. And it's another input is a set of cuboids recommended by cube planner algorithms. By comparing the new recommended cuboids _*R*_ and current prebuilt cuboids *C*, optimize job will do the following things: # Filter the overlap cuboids and remove the ones in *C* but not in *R* > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids *C*, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in *C* but not in *R* -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2733: -- Description: Optimize job is for adjusting existing cuboid set. Its data source is base cuboid. And it's another input is a set of cuboids recommended by cube planner algorithms. By comparing the new recommended cuboids _*R*_ and current prebuilt cuboids _*C*_, optimize job will do the following things: # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ # Update the shard id of the overlap cuboids # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input for cubing was: Optimize job is for adjusting existing cuboid set. Its data source is base cuboid. And it's another input is a set of cuboids recommended by cube planner algorithms. By comparing the new recommended cuboids _*R*_ and current prebuilt cuboids *C*, optimize job will do the following things: # Filter the overlap cuboids and remove the ones in *C* but not in *R* > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Optimize job is for adjusting existing cuboid set. Its data source is base > cuboid. And it's another input is a set of cuboids recommended by cube > planner algorithms. By comparing the new recommended cuboids _*R*_ and > current prebuilt cuboids _*C*_, optimize job will do the following things: > # Filter the overlap cuboids and remove the ones in _*C*_ but not in _*R*_ > # Update the shard id of the overlap cuboids > # Add the missing cuboids in _*R*_ but not in _*C*_ with base cuboid as input > for cubing -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[
https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2731:
--
Description:
The basic idea of cube optimization without influencing current queries is
shown at figure [^One-Click Cube Optimization.png].
To optimize a cube with N ready segments,
* For each segment, a corresponding optimize job will be triggered to adjust
its cuboids.
* If a optimize job finished, it's resulting segment status should be
READY_PENDING and will not serve query.
* There should be job to wait for all of the optimize jobs finishing their
work, and then atomically remove old READY segments and update the status of
READY_PENDING segments to READY for serving query. Here, we call this kind of
job as {color:#f79232}checkpoint executable{color}.
was:
The basic idea of cube optimization without influencing current queries is
shown at figure [^One-Click Cube Optimization.png].
To optimize a cube with N ready segments,
* For each segment, a corresponding optimize job will be triggered to adjust
its cuboids.
* If a optimize job finished, it's resulting segment status should be
READY_PENDING and will not serve query.
* There should be job to wait for all of the optimize jobs finishing their
work, and then atomically remove old READY segments and update the status of
READY_PENDING segments to READY for serving query.
> Introduce checkpoint executable
> ---
>
> Key: KYLIN-2731
> URL: https://issues.apache.org/jira/browse/KYLIN-2731
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: One-Click Cube Optimization.png
>
>
> The basic idea of cube optimization without influencing current queries is
> shown at figure [^One-Click Cube Optimization.png].
> To optimize a cube with N ready segments,
> * For each segment, a corresponding optimize job will be triggered to adjust
> its cuboids.
> * If a optimize job finished, it's resulting segment status should be
> READY_PENDING and will not serve query.
> * There should be job to wait for all of the optimize jobs finishing their
> work, and then atomically remove old READY segments and update the status of
> READY_PENDING segments to READY for serving query. Here, we call this kind of
> job as {color:#f79232}checkpoint executable{color}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2731: -- Description: The basic idea of cube optimization without influencing current queries is shown at figure [^One-Click Cube Optimization.png]. To optimize a cube with N ready segments, * For each segment, a corresponding optimize job will be triggered to adjust its cuboids. * If a optimize job finished, it's resulting segment status should be READY_PENDING and will not serve query. * There should be job to wait for all of the optimize jobs finishing their work, and then atomically remove old READY segments and update the status of READY_PENDING segments to READY for serving query. was: The basic idea of cube optimization without influencing current queries is shown at figure [^One-Click Cube Optimization.png]. To optimize a cube with N ready segments, * For each segment, a corresponding optimize job will be triggered to adjust its cuboids. * If a optimize job finished, it's resulting segment status should be READY_PENDING > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: One-Click Cube Optimization.png > > > The basic idea of cube optimization without influencing current queries is > shown at figure [^One-Click Cube Optimization.png]. > To optimize a cube with N ready segments, > * For each segment, a corresponding optimize job will be triggered to adjust > its cuboids. > * If a optimize job finished, it's resulting segment status should be > READY_PENDING and will not serve query. > * There should be job to wait for all of the optimize jobs finishing their > work, and then atomically remove old READY segments and update the status of > READY_PENDING segments to READY for serving query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2731: -- Description: The basic idea of cube optimization without influencing current queries is shown at figure [^One-Click Cube Optimization.png]. To optimize a cube with N ready segments, * For each segment, a corresponding optimize job will be triggered to adjust its cuboids. * If a optimize job finished, it's resulting segment status should be READY_PENDING was:The basic idea of cube optimization without influencing current queries is shown at figure [^One-Click Cube Optimization.png] > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: One-Click Cube Optimization.png > > > The basic idea of cube optimization without influencing current queries is > shown at figure [^One-Click Cube Optimization.png]. > To optimize a cube with N ready segments, > * For each segment, a corresponding optimize job will be triggered to adjust > its cuboids. > * If a optimize job finished, it's resulting segment status should be > READY_PENDING -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2731: -- Description: The basic idea of cube optimization without influencing current queries is shown at figure [^One-Click Cube Optimization.png] (was: The basic idea of cube optimization without influencing current queries is shown at figure [^]) > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: One-Click Cube Optimization.png > > > The basic idea of cube optimization without influencing current queries is > shown at figure [^One-Click Cube Optimization.png] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2731: -- Description: The basic idea of cube optimization without influencing current queries is shown at figure [^] > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: One-Click Cube Optimization.png > > > The basic idea of cube optimization without influencing current queries is > shown at figure [^] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2731: -- Attachment: One-Click Cube Optimization.png > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: One-Click Cube Optimization.png > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2732: -- Description: If the design for an existing cube is not good, which means its prebuilt cuboid set is not suitable for query, there should be a way to adjust this cuboid set. This is phase two of cube planner, also called cube optimization. Its data source should be base cuboid rather than the original data source, like hive, for the original one may be changed. Thus for cubing jobs, like cube optimization, we introduce a new input , base cuboid formatted by sequence file. (was: If the design for an existing cube is not good, which means its prebuilt cuboid set is not suitable for query, there should be a way to adjust this cuboid set. Here, we call For cube optimization, cuboids to be pre-calculated need to be adjusted. During this phase, there are three kinds of cuboids: # cuboids to be kept # cuboids to be removed # cuboids to be added The *cuboids to be added* can be derived from the base cuboid. Thus for cubing job, we introduce a new input , base cuboid formatted by sequence file) > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. This is phase two of cube planner, also called cube optimization. > Its data source should be base cuboid rather than the original data source, > like hive, for the original one may be changed. Thus for cubing jobs, like > cube optimization, we introduce a new input , base cuboid formatted by > sequence file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2732) Introduce base cuboid as a new input for cubing job
[ https://issues.apache.org/jira/browse/KYLIN-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2732: -- Description: If the design for an existing cube is not good, which means its prebuilt cuboid set is not suitable for query, there should be a way to adjust this cuboid set. Here, we call For cube optimization, cuboids to be pre-calculated need to be adjusted. During this phase, there are three kinds of cuboids: # cuboids to be kept # cuboids to be removed # cuboids to be added The *cuboids to be added* can be derived from the base cuboid. Thus for cubing job, we introduce a new input , base cuboid formatted by sequence file was: For cube optimization, cuboids to be pre-calculated need to be adjusted. During this phase, there are three kinds of cuboids: # cuboids to be kept # cuboids to be removed # cuboids to be added The *cuboids to be added* can be derived from the base cuboid. Thus for cubing job, we introduce a new input , base cuboid formatted by sequence file > Introduce base cuboid as a new input for cubing job > --- > > Key: KYLIN-2732 > URL: https://issues.apache.org/jira/browse/KYLIN-2732 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > If the design for an existing cube is not good, which means its prebuilt > cuboid set is not suitable for query, there should be a way to adjust this > cuboid set. Here, we call > For cube optimization, cuboids to be pre-calculated need to be adjusted. > During this phase, there are three kinds of cuboids: > # cuboids to be kept > # cuboids to be removed > # cuboids to be added > The *cuboids to be added* can be derived from the base cuboid. Thus for > cubing job, we introduce a new input , base cuboid formatted by sequence file -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2802) Enable cube planner phase one
[ https://issues.apache.org/jira/browse/KYLIN-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2802: -- Description: This phase happens at cube building. To trigger cube planner, one of the following conditions should be satisfied. * The cube is empty * Job is for refreshing the only one segment. Since cube planner is based on cuboid row count, if the first segment is empty and there's no useful row count info, cube planner should not be triggered in this phase. > Enable cube planner phase one > - > > Key: KYLIN-2802 > URL: https://issues.apache.org/jira/browse/KYLIN-2802 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Fix For: v2.2.0 > > > This phase happens at cube building. To trigger cube planner, one of the > following conditions should be satisfied. > * The cube is empty > * Job is for refreshing the only one segment. > Since cube planner is based on cuboid row count, if the first segment is > empty and there's no useful row count info, cube planner should not be > triggered in this phase. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2730) Introduce genetic algorithm for cube planner
[ https://issues.apache.org/jira/browse/KYLIN-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2730: -- Description: The basic idea of this algorithm is to evolve the generation of chromosome, where a chromosome is a cuboid set to prebuilt. The detailed steps are as follows: # Initialize a generation of chromosome # Evolve the generation by selection, crossover and mutation by N round. # From the final generation, choose a best chromosome that is a best cuboid set to recommend. > Introduce genetic algorithm for cube planner > > > Key: KYLIN-2730 > URL: https://issues.apache.org/jira/browse/KYLIN-2730 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > The basic idea of this algorithm is to evolve the generation of chromosome, > where a chromosome is a cuboid set to prebuilt. The detailed steps are as > follows: > # Initialize a generation of chromosome > # Evolve the generation by selection, crossover and mutation by N round. > # From the final generation, choose a best chromosome that is a best cuboid > set to recommend. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2729) Introduce greedy algorithm for cube planner
[ https://issues.apache.org/jira/browse/KYLIN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2729: -- Description: The basic idea of greedy algorithm is to select the highest benefit cuboid round by round. Notice that the result got from this algorithm is not the global optimization one > Introduce greedy algorithm for cube planner > --- > > Key: KYLIN-2729 > URL: https://issues.apache.org/jira/browse/KYLIN-2729 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > The basic idea of greedy algorithm is to select the highest benefit cuboid > round by round. Notice that the result got from this algorithm is not the > global optimization one -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2826) Add basic support classes for cube planner algorithms
[
https://issues.apache.org/jira/browse/KYLIN-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2826:
--
Description:
Cube planner aims at recommending cost-effective cuboids. Currently we only
consider {color:#f79232}*scanned row count*{color} at {color:#f79232}*query
phase*{color} for the cost. The related formula is as follows:
bq. cuboid cost = scanned row count on target cuboid * query probability
As we know the base cuboid is to be prebuilt absolutely. If only the base
cuboid is prebuilt, for other cuboids, the target cuboid will be the base
cuboid and the _(scanned row count)_ is supposed to be large. When another
cuboid is selected to be prebuilt, for its descendant cuboids including itself,
it will be their target cuboid and the _(scanned row count)_ is supposed to
become smaller. Thus, this newly cuboid will bring some benefit. We employ BPUS
(benefit per unit space) for cuboid selection. The related formula for the
benefit of a cuboid is as follows:
bq. cuboid benefit = total reduced cuboid cost) / (cuboid row count)
Cuboid selection is based on one basic rule:
bq. {color:#f79232}*RULE 1: Cuboids with more benefit will be preferred.*{color}
For a cube, cube planner can be used in two phases.
* Phase one is for cube normal building.
To use cube planner for this phase, the cube should be empty or the building
job is for refreshing the only one segment. In this phase, we regard each
cuboid own the same _(query probability)_ due to lack of query statistics.
* Phase two is for cube optimization.
Currently cube optimization is manually triggered. _(query probability)_ will
be considered and its related query statistics are fetched from system cubes.
Based on _(query probability)_, it's possible for us to add missing cuboids
without cuboid row count info. It's based on a rule, called
{color:#f79232}*mandatory rule*{color}.
bq. {color:#f79232}*RULE 2: A cuboid not pre-built should be added, if it's
queried frequently and the average rollup row count from its pre-built parent
cuboid is large.*{color}
>From above introduction, we know cube planner is based on statistics,
>including cuboid row count, cuboid hit frequency, etc. Class {{CuboidStats}}
>is introduced to provide these info for related algorithm.
Here, we also define the interface {{CuboidRecommendAlgorithm}} for different
kinds of cube planner algorithms. As we know, if there's no space limitation,
to pre-build all of the cuboids will bring the most benefit. However, it's not
feasible in real world. Then with space limitation, an interface is defined to
recommend a set of high benefit cuboids.
{code}
List recommend(double expansionRate);
{code}
Here, the expansion rate is compared to the size of base cuboid.
was:
Cube planner aims at recommending cost-effective cuboids. Currently we only
consider {color:#f79232}*scanned row count*{color} at {color:#f79232}*query
phase*{color} for the cost. The related formula is as follows:
bq. cuboid cost = scanned row count on target cuboid * query probability
As we know the base cuboid is to be prebuilt absolutely. If only the base
cuboid is prebuilt, for other cuboids, the target cuboid will be the base
cuboid and the _(scanned row count)_ is supposed to be large. When another
cuboid is selected to be prebuilt, for its descendant cuboids including itself,
it will be their target cuboid and the _(scanned row count)_ is supposed to
become smaller. Thus, this newly cuboid will bring some benefit. We employ BPUS
(benefit per unit space) for cuboid selection. The related formula for the
benefit of a cuboid is as follows:
bq. cuboid benefit = total reduced cuboid cost) / (cuboid row count)
Cuboid selection is based on one basic rule:
bq. {color:#f79232}*RULE: Cuboids with more benefit will be preferred.*{color}
> Add basic support classes for cube planner algorithms
> -
>
> Key: KYLIN-2826
> URL: https://issues.apache.org/jira/browse/KYLIN-2826
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> Cube planner aims at recommending cost-effective cuboids. Currently we only
> consider {color:#f79232}*scanned row count*{color} at {color:#f79232}*query
> phase*{color} for the cost. The related formula is as follows:
> bq. cuboid cost = scanned row count on target cuboid * query probability
> As we know the base cuboid is to be prebuilt absolutely. If only the base
> cuboid is prebuilt, for other cuboids, the target cuboid will be the base
> cuboid and the _(scanned row count)_ is supposed to be large. When another
> cuboid is selected to be prebuilt, for its descendant cuboids including
> itself, it will be their target cuboid and the _(scanned row count)_ is
> supposed to become smaller. Thus, this
[jira] [Created] (KYLIN-2829) Union two subquery under join make SQL return null values
Roger Shi created KYLIN-2829:
Summary: Union two subquery under join make SQL return null values
Key: KYLIN-2829
URL: https://issues.apache.org/jira/browse/KYLIN-2829
Project: Kylin
Issue Type: Bug
Reporter: Roger Shi
SQL like:
{code:sql}
select sum(TEST_A.PRICE) as ITEM_CNT
FROM TEST_KYLIN_FACT as TEST_A
join (
select * from ITEM_CNT where CAL_DT < DATE '2012-06-01'
union
select * from ITEM_CNT where CAL_DT > DATE '2013-06-01'
) TEST_B
on TEST_A.TRANS_ID = TEST_B.TRANS_ID
group by TEST_A.SELLER_ID
{code}
returns null values.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-2830) Union two subquery under join make SQL return null values
Roger Shi created KYLIN-2830:
Summary: Union two subquery under join make SQL return null values
Key: KYLIN-2830
URL: https://issues.apache.org/jira/browse/KYLIN-2830
Project: Kylin
Issue Type: Bug
Reporter: Roger Shi
SQL like:
{code:sql}
select sum(TEST_A.PRICE) as ITEM_CNT
FROM TEST_KYLIN_FACT as TEST_A
join (
select * from ITEM_CNT where CAL_DT < DATE '2012-06-01'
union
select * from ITEM_CNT where CAL_DT > DATE '2013-06-01'
) TEST_B
on TEST_A.TRANS_ID = TEST_B.TRANS_ID
group by TEST_A.SELLER_ID
{code}
returns null values.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-2828) Project meta should be involved for Spark Cubing
Wang Cheng created KYLIN-2828: - Summary: Project meta should be involved for Spark Cubing Key: KYLIN-2828 URL: https://issues.apache.org/jira/browse/KYLIN-2828 Project: Kylin Issue Type: Bug Reporter: Wang Cheng Assignee: Wang Cheng Priority: Minor java.lang.IllegalStateException: Error to load /model_desc/ci_inner_join_model.json at org.apache.kylin.metadata.MetadataManager.reloadDataModelDescAt(MetadataManager.java:633) at org.apache.kylin.metadata.MetadataManager.reloadAllDataModel(MetadataManager.java:611) at org.apache.kylin.metadata.MetadataManager.init(MetadataManager.java:323) at org.apache.kylin.metadata.MetadataManager.(MetadataManager.java:136) at org.apache.kylin.metadata.MetadataManager.getInstance(MetadataManager.java:94) at org.apache.kylin.cube.model.CubeDesc.init(CubeDesc.java:589) at org.apache.kylin.cube.CubeDescManager.loadCubeDesc(CubeDescManager.java:196) at org.apache.kylin.cube.CubeDescManager.reloadAllCubeDesc(CubeDescManager.java:321) at org.apache.kylin.cube.CubeDescManager.(CubeDescManager.java:114) at org.apache.kylin.cube.CubeDescManager.getInstance(CubeDescManager.java:81) at org.apache.kylin.cube.CubeManager.reloadCubeLocalAt(CubeManager.java:809) at org.apache.kylin.cube.CubeManager.loadAllCubeInstance(CubeManager.java:787) at org.apache.kylin.cube.CubeManager.(CubeManager.java:147) at org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:105) at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:295) at org.apache.kylin.engine.spark.SparkCubingByLayer$EncodeBaseCuboid.call(SparkCubingByLayer.java:276) at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043) at scala.collection.Iterator$$anon$11.next(Iterator.scala:409) at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:193) at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:63) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2826) Add basic support classes for cube planner algorithms
[
https://issues.apache.org/jira/browse/KYLIN-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2826:
--
Description:
Cube planner aims at recommending cost-effective cuboids. Currently we only
consider {color:#f79232}*scanned row count*{color} at {color:#f79232}*query
phase*{color} for the cost. The related formula is as follows:
bq. cuboid cost = scanned row count on target cuboid * query probability
As we know the base cuboid is to be prebuilt absolutely. If only the base
cuboid is prebuilt, for other cuboids, the target cuboid will be the base
cuboid and the _(scanned row count)_ is supposed to be large. When another
cuboid is selected to be prebuilt, for its descendant cuboids including itself,
it will be their target cuboid and the _(scanned row count)_ is supposed to
become smaller. Thus, this newly cuboid will bring some benefit. We employ BPUS
(benefit per unit space) for cuboid selection. The related formula for the
benefit of a cuboid is as follows:
bq. cuboid benefit = total reduced cuboid cost) / (cuboid row count)
Cuboid selection is based on one basic rule:
bq. {color:#f79232}*RULE: Cuboids with more benefit will be preferred.*{color}
> Add basic support classes for cube planner algorithms
> -
>
> Key: KYLIN-2826
> URL: https://issues.apache.org/jira/browse/KYLIN-2826
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> Cube planner aims at recommending cost-effective cuboids. Currently we only
> consider {color:#f79232}*scanned row count*{color} at {color:#f79232}*query
> phase*{color} for the cost. The related formula is as follows:
> bq. cuboid cost = scanned row count on target cuboid * query probability
> As we know the base cuboid is to be prebuilt absolutely. If only the base
> cuboid is prebuilt, for other cuboids, the target cuboid will be the base
> cuboid and the _(scanned row count)_ is supposed to be large. When another
> cuboid is selected to be prebuilt, for its descendant cuboids including
> itself, it will be their target cuboid and the _(scanned row count)_ is
> supposed to become smaller. Thus, this newly cuboid will bring some benefit.
> We employ BPUS (benefit per unit space) for cuboid selection. The related
> formula for the benefit of a cuboid is as follows:
> bq. cuboid benefit = total reduced cuboid cost) / (cuboid row count)
> Cuboid selection is based on one basic rule:
> bq. {color:#f79232}*RULE: Cuboids with more benefit will be preferred.*{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianhao Zhao updated KYLIN-2824: Attachment: (was: KYLIN-2824.patch) > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianhao Zhao updated KYLIN-2824: Attachment: KYLIN-2824.patch > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch, KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2827) cube已经建好了，再insight 查询回报错
liuxd created KYLIN-2827: Summary: cube已经建好了，再insight 查询回报错 Key: KYLIN-2827 URL: https://issues.apache.org/jira/browse/KYLIN-2827 Project: Kylin Issue Type: Bug Reporter: liuxd Error while executing SQL "select * from GDL_LOANRPY_DET LIMIT 5": org.apache.hadoop.hbase.exceptions.UnknownProtocolException: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered coprocessor service found for name CubeVisitService in region KYLIN_ZW5LTEKVAL,,1504166387142.432256ac271b98fc6afd29bc7e054502. at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7582) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1890) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1872) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2720) Should not allow user to access to all tables' metadata of a project
[ https://issues.apache.org/jira/browse/KYLIN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148686#comment-16148686 ] qiumingming commented on KYLIN-2720: Okay, I see. But I'm also confused. Is that means I will not see any some other tables in the web UI that I owned cubes do not dependent to. I have not upgraded to kylin 2.1, so I want to know it. As you can see, my patch did nothing about ACL grant or revoke, it just control tables that being displayed in the web UI. So user will not see tables he can not query to in the web UI. Otherwise, user will be confused : why display tables that I can not query? > Should not allow user to access to all tables' metadata of a project > > > Key: KYLIN-2720 > URL: https://issues.apache.org/jira/browse/KYLIN-2720 > Project: Kylin > Issue Type: Improvement >Reporter: qiumingming >Assignee: qiumingming > Attachments: KYLIN-2720.patch > > > Currently, user can access to all tables and columns metadata of a specific > project as long as he can access to this project, which is not reasonable. > User should just allow to access to tables that he owned cubes dependent to. > However, user can see some other tables in the web UI in current version. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2720) Should not allow user to access to all tables' metadata of a project
[ https://issues.apache.org/jira/browse/KYLIN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148633#comment-16148633 ] Shaofeng SHI commented on KYLIN-2720: - Hi qiumingming, Sorry for the late; As Hongbin Ma commented above, this change is conflict with KYLIN-2646 (user can manage the ACL at project/table level, not cube level), so we're not going to merge this patch; If you need this change for your version, you can keep the patch in local. > Should not allow user to access to all tables' metadata of a project > > > Key: KYLIN-2720 > URL: https://issues.apache.org/jira/browse/KYLIN-2720 > Project: Kylin > Issue Type: Improvement >Reporter: qiumingming >Assignee: qiumingming > Attachments: KYLIN-2720.patch > > > Currently, user can access to all tables and columns metadata of a specific > project as long as he can access to this project, which is not reasonable. > User should just allow to access to tables that he owned cubes dependent to. > However, user can see some other tables in the web UI in current version. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2800) All dictionaries should be built based on the flat hive table
[ https://issues.apache.org/jira/browse/KYLIN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148634#comment-16148634 ] zhengdong commented on KYLIN-2800: -- Hi liyang, merging lookup dictionaries is indeed a relatively easier solution. However, if these lookup dictionaries always need merged, uniformly using the flat hive table as dictionaries data source should has better performance, especially for large lookup tables, and makes it convenient to build all dictionaries concurrently in reducers. > All dictionaries should be built based on the flat hive table > - > > Key: KYLIN-2800 > URL: https://issues.apache.org/jira/browse/KYLIN-2800 > Project: Kylin > Issue Type: Bug >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2800-All-dictionaries-should-be-built-based-on.patch > > > After KYLIN-2457, we still got wrong query result sometimes after a merging > job finished. > Finally, we realize the root cause is that we always use lookup table as > source data to build dictionaries for FK columns. > However, incremental lookup table doesn't mean sequential and incremental PK. > If a new record inserted into the lookup table while its PK column does not > have the max value, ID numbers in the new dictionary could be changed for > those PK value larger than the newest one. What's more, using lookup table as > source data for FK column's dictionary may has performance advantage for > merging job, but also may encounter too big dictionary problem for large > lookup tables. And we must add some validation rules to ensure the PK value > sequential and incremental. > On the another hand, we could just unify using the flat hive table as data > source for all dictionaries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2729) Introduce greedy algorithm for cube planner
[ https://issues.apache.org/jira/browse/KYLIN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136616#comment-16136616 ] Zhong Yanghong edited comment on KYLIN-2729 at 8/31/17 7:44 AM: Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/913e97594e7b9ae03bb7348a0f092770f736814d was (Author: yaho): Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/7c592b3c33d1af3a933af0b77c154b250e320bb8 https://github.com/apache/kylin/commit/913e97594e7b9ae03bb7348a0f092770f736814d > Introduce greedy algorithm for cube planner > --- > > Key: KYLIN-2729 > URL: https://issues.apache.org/jira/browse/KYLIN-2729 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2826) Add basic support classes for cube planner algorithms
[ https://issues.apache.org/jira/browse/KYLIN-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148586#comment-16148586 ] Zhong Yanghong commented on KYLIN-2826: --- Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/7c592b3c33d1af3a933af0b77c154b250e320bb8 > Add basic support classes for cube planner algorithms > - > > Key: KYLIN-2826 > URL: https://issues.apache.org/jira/browse/KYLIN-2826 > Project: Kylin > Issue Type: Sub-task >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2826) Add basic support classes for cube planner algorithms
Zhong Yanghong created KYLIN-2826: - Summary: Add basic support classes for cube planner algorithms Key: KYLIN-2826 URL: https://issues.apache.org/jira/browse/KYLIN-2826 Project: Kylin Issue Type: Sub-task Reporter: Zhong Yanghong Assignee: Zhong Yanghong -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2728) Introduce a new cuboid scheduler based on cuboid tree rather than static rules
[
https://issues.apache.org/jira/browse/KYLIN-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2728:
--
Description:
Rather than depending on static rules to define a parent-child relationship
between cuboids, a cuboid tree can be constructed for this relationship. In
this tree, the root is the base cuboid. For a cuboid, it may have many legal
parent cuboids. We should choose the {color:#f79232}best parent {color}cuboid
as its parent in this tree. Then how to define the best? It's related two rules
as follows:
* If we know the row count, choose the one with the least row count
* Otherwise, choose the smallest cuboid
Based on the cuboid tree, we introduced a new cuboid scheduler called
{{TreeCuboidScheduler}}. And both cube building & query can be based on
{{TreeCuboidScheduler}}.
> Introduce a new cuboid scheduler based on cuboid tree rather than static rules
> --
>
> Key: KYLIN-2728
> URL: https://issues.apache.org/jira/browse/KYLIN-2728
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> Rather than depending on static rules to define a parent-child relationship
> between cuboids, a cuboid tree can be constructed for this relationship. In
> this tree, the root is the base cuboid. For a cuboid, it may have many legal
> parent cuboids. We should choose the {color:#f79232}best parent {color}cuboid
> as its parent in this tree. Then how to define the best? It's related two
> rules as follows:
> * If we know the row count, choose the one with the least row count
> * Otherwise, choose the smallest cuboid
> Based on the cuboid tree, we introduced a new cuboid scheduler called
> {{TreeCuboidScheduler}}. And both cube building & query can be based on
> {{TreeCuboidScheduler}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2822) Introduce sunburst chart to show cuboid tree
[ https://issues.apache.org/jira/browse/KYLIN-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147019#comment-16147019 ] Zhong Yanghong edited comment on KYLIN-2822 at 8/31/17 7:13 AM: Related commit for branch yaho-cube-planner is as follows: * Backend: https://github.com/apache/kylin/commit/0b12df776348c7863bea6363ceacfe7380cb4e15 was (Author: yaho): Backend: https://github.com/apache/kylin/commit/0b12df776348c7863bea6363ceacfe7380cb4e15 > Introduce sunburst chart to show cuboid tree > > > Key: KYLIN-2822 > URL: https://issues.apache.org/jira/browse/KYLIN-2822 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2734) Introduce hot cuboids export & import
[ https://issues.apache.org/jira/browse/KYLIN-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147022#comment-16147022 ] Zhong Yanghong edited comment on KYLIN-2734 at 8/31/17 7:13 AM: Related commit for branch yaho-cube-planner is as follows: * Backend: https://github.com/apache/kylin/commit/7279427d40df7e9d6151ecf3f439c2c20f800dd1 was (Author: yaho): Backend: https://github.com/apache/kylin/commit/7279427d40df7e9d6151ecf3f439c2c20f800dd1 > Introduce hot cuboids export & import > - > > Key: KYLIN-2734 > URL: https://issues.apache.org/jira/browse/KYLIN-2734 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148547#comment-16148547
]
Zhong Yanghong edited comment on KYLIN-2735 at 8/31/17 7:13 AM:
Related commit for branch yaho-cube-planner is as follows:
https://github.com/apache/kylin/commit/d216eba3e10af99cb58b9ca45582d2bf5c8a0a37
was (Author: yaho):
https://github.com/apache/kylin/commit/d216eba3e10af99cb58b9ca45582d2bf5c8a0a37
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** {color:#f79232}first check the job priority queue and schedule jobs
> meeting condition{color} {color:#f691b2}*C*{color}
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition {color:#f691b2}*C*{color} is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2733) Introduce optimize job for adjusting cuboid set
[ https://issues.apache.org/jira/browse/KYLIN-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146562#comment-16146562 ] Zhong Yanghong edited comment on KYLIN-2733 at 8/31/17 7:12 AM: Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/0c4b3ad57277653e2b360c0acbf1c9c2b6d4d6f2 was (Author: yaho): https://github.com/apache/kylin/commit/0c4b3ad57277653e2b360c0acbf1c9c2b6d4d6f2 > Introduce optimize job for adjusting cuboid set > --- > > Key: KYLIN-2733 > URL: https://issues.apache.org/jira/browse/KYLIN-2733 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2731) Introduce checkpoint executable
[ https://issues.apache.org/jira/browse/KYLIN-2731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141110#comment-16141110 ] Zhong Yanghong edited comment on KYLIN-2731 at 8/31/17 7:11 AM: Related commit for branch yaho-cube-planner is as follows: https://github.com/apache/kylin/commit/7603a5aafa64e990e3f8a40c267f835d274aef61 was (Author: yaho): https://github.com/apache/kylin/commit/7603a5aafa64e990e3f8a40c267f835d274aef61 > Introduce checkpoint executable > --- > > Key: KYLIN-2731 > URL: https://issues.apache.org/jira/browse/KYLIN-2731 > Project: Kylin > Issue Type: Sub-task >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148547#comment-16148547
]
Zhong Yanghong commented on KYLIN-2735:
---
https://github.com/apache/kylin/commit/d216eba3e10af99cb58b9ca45582d2bf5c8a0a37
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** {color:#f79232}first check the job priority queue and schedule jobs
> meeting condition{color} {color:#f691b2}*C*{color}
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition {color:#f691b2}*C*{color} is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148548#comment-16148548 ] Billy Liu commented on KYLIN-2824: -- Thanks the patch. LGTM. > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2735:
--
Description:
To optimize a cube with *N* ready segments, *N* optimize jobs + *1* checkpoint
job are needed, which may occupy too much resources and make other jobs waiting
too much time to be scheduled. Thus, job priority and its dynamic changing will
be introduced. The idea is as follows:
* Job Fetcher fetches job metadata from database every *M* seconds.
* A Running Job List is maintained globally
* {color:#f79232}A Job Priority Queue is maintained globally{color}
* Before fetching job metadata,
** {color:#f79232}first check the job priority queue and schedule jobs meeting
condition{color} {color:#f691b2}*C*{color}
** then if the size of Running Job List exceeds the max concurrent job limit
*L*, this fetch round will be skipped
* During fetch round,
** {color:#f79232} the priority of each job in the queue will increase.{color}
** {color:#f79232} each new job fetched will be assigned with a default
priority.{color}
** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
List and be submitted to Job Execute Thread Pool
* Once a job finishes, it will be removed from Running Job List and notify the
Job Fetcher to fetch job metadata
The condition {color:#f691b2}*C*{color} is as follows:
* job priority should be above a threshold, to filter low priority jobs
* job should wait at least once, to give chance for low priority jobs to be
scheduled
was:
To optimize a cube with *N* ready segments, *N* optimize jobs + *1* checkpoint
job are needed, which may occupy too much resources and make other jobs waiting
too much time to be scheduled. Thus, job priority and its dynamic changing will
be introduced. The idea is as follows:
* Job Fetcher fetches job metadata from database every *M* seconds.
* A Running Job List is maintained globally
* {color:#f79232}A Job Priority Queue is maintained globally{color}
* Before fetching job metadata,
** {color:#f79232}first check the job priority queue and schedule jobs meeting
condition *C*{color}
** then if the size of Running Job List exceeds the max concurrent job limit
*L*, this fetch round will be skipped
* During fetch round,
** {color:#f79232} the priority of each job in the queue will increase.{color}
** {color:#f79232} each new job fetched will be assigned with a default
priority.{color}
** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
List and be submitted to Job Execute Thread Pool
* Once a job finishes, it will be removed from Running Job List and notify the
Job Fetcher to fetch job metadata
The condition *C* is as follows:
* job priority should be above a threshold, to filter low priority jobs
* job should wait at least once, to give chance for low priority jobs to be
scheduled
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** {color:#f79232}first check the job priority queue and schedule jobs
> meeting condition{color} {color:#f691b2}*C*{color}
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition {color:#f691b2}*C*{color} is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2735:
--
Description:
To optimize a cube with *N* ready segments, *N* optimize jobs + *1* checkpoint
job are needed, which may occupy too much resources and make other jobs waiting
too much time to be scheduled. Thus, job priority and its dynamic changing will
be introduced. The idea is as follows:
* Job Fetcher fetches job metadata from database every *M* seconds.
* A Running Job List is maintained globally
* {color:#f79232}A Job Priority Queue is maintained globally{color}
* Before fetching job metadata,
** {color:#f79232}first check the job priority queue and schedule jobs meeting
condition *C*{color}
** then if the size of Running Job List exceeds the max concurrent job limit
*L*, this fetch round will be skipped
* During fetch round,
** {color:#f79232} the priority of each job in the queue will increase.{color}
** {color:#f79232} each new job fetched will be assigned with a default
priority.{color}
** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
List and be submitted to Job Execute Thread Pool
* Once a job finishes, it will be removed from Running Job List and notify the
Job Fetcher to fetch job metadata
The condition *C* is as follows:
* job priority should be above a threshold, to filter low priority jobs
* job should wait at least once, to give chance for low priority jobs to be
scheduled
was:
To optimize a cube with *N* ready segments, *N* optimize jobs + *1* checkpoint
job are needed, which may occupy too much resources and make other jobs waiting
too much time to be scheduled. Thus, job priority and its dynamic changing will
be introduced. The idea is as follows:
* Job Fetcher fetches job metadata from database every *M* seconds.
* A Running Job List is maintained globally
* {color:#f79232}A Job Priority Queue is maintained globally{color}
* Before fetching job metadata,
** first check the job priority queue and schedule jobs meeting condition *C*
** then if the size of Running Job List exceeds the max concurrent job limit
*L*, this fetch round will be skipped
* During fetch round,
** {color:#f79232} the priority of each job in the queue will increase.{color}
** {color:#f79232} each new job fetched will be assigned with a default
priority.{color}
** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
List and be submitted to Job Execute Thread Pool
* Once a job finishes, it will be removed from Running Job List and notify the
Job Fetcher to fetch job metadata
The condition *C* is as follows:
* job priority should be above a threshold, to filter low priority jobs
* job should wait at least once, to give chance for low priority jobs to be
scheduled
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** {color:#f79232}first check the job priority queue and schedule jobs
> meeting condition *C*{color}
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition *C* is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[
https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhong Yanghong updated KYLIN-2735:
--
Description:
To optimize a cube with *N* ready segments, *N* optimize jobs + *1* checkpoint
job are needed, which may occupy too much resources and make other jobs waiting
too much time to be scheduled. Thus, job priority and its dynamic changing will
be introduced. The idea is as follows:
* Job Fetcher fetches job metadata from database every *M* seconds.
* A Running Job List is maintained globally
* {color:#f79232}A Job Priority Queue is maintained globally{color}
* Before fetching job metadata,
** first check the job priority queue and schedule jobs meeting condition *C*
** then if the size of Running Job List exceeds the max concurrent job limit
*L*, this fetch round will be skipped
* During fetch round,
** {color:#f79232} the priority of each job in the queue will increase.{color}
** {color:#f79232} each new job fetched will be assigned with a default
priority.{color}
** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
List and be submitted to Job Execute Thread Pool
* Once a job finishes, it will be removed from Running Job List and notify the
Job Fetcher to fetch job metadata
The condition *C* is as follows:
* job priority should be above a threshold, to filter low priority jobs
* job should wait at least once, to give chance for low priority jobs to be
scheduled
> Introduce an option to make job scheduler consider job priority
> ---
>
> Key: KYLIN-2735
> URL: https://issues.apache.org/jira/browse/KYLIN-2735
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>
> To optimize a cube with *N* ready segments, *N* optimize jobs + *1*
> checkpoint job are needed, which may occupy too much resources and make other
> jobs waiting too much time to be scheduled. Thus, job priority and its
> dynamic changing will be introduced. The idea is as follows:
> * Job Fetcher fetches job metadata from database every *M* seconds.
> * A Running Job List is maintained globally
> * {color:#f79232}A Job Priority Queue is maintained globally{color}
> * Before fetching job metadata,
> ** first check the job priority queue and schedule jobs meeting condition *C*
> ** then if the size of Running Job List exceeds the max concurrent job limit
> *L*, this fetch round will be skipped
> * During fetch round,
> ** {color:#f79232} the priority of each job in the queue will increase.{color}
> ** {color:#f79232} each new job fetched will be assigned with a default
> priority.{color}
> ** {color:#f79232} the highest *L*{color} jobs will be added to Running Job
> List and be submitted to Job Execute Thread Pool
> * Once a job finishes, it will be removed from Running Job List and notify
> the Job Fetcher to fetch job metadata
> The condition *C* is as follows:
> * job priority should be above a threshold, to filter low priority jobs
> * job should wait at least once, to give chance for low priority jobs to be
> scheduled
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-2825) Make "ModelDataGenerator.java" serilizable
Shaofeng SHI created KYLIN-2825: --- Summary: Make "ModelDataGenerator.java" serilizable Key: KYLIN-2825 URL: https://issues.apache.org/jira/browse/KYLIN-2825 Project: Kylin Issue Type: Improvement Components: Tools, Build and Test Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor The ModelDataGenerator was used for mockup test data. Now it can only run in single VM. If it can run in parallel, then will generate more data. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148530#comment-16148530 ] Tianhao Zhao commented on KYLIN-2824: - patch attached > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianhao Zhao updated KYLIN-2824: Attachment: KYLIN-2824.patch > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2824) Parse Boolean type in JDBC driver
Tianhao Zhao created KYLIN-2824: --- Summary: Parse Boolean type in JDBC driver Key: KYLIN-2824 URL: https://issues.apache.org/jira/browse/KYLIN-2824 Project: Kylin Issue Type: Bug Components: Driver - JDBC Reporter: Tianhao Zhao Priority: Minor Exception below is thrown when querying kylin with Saiku as front end, or when retrieving value from getBoolean method in jdbc driver. That's because KylinClient in jdbc module dose not parse value properly when dimension is boolean type. Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) at org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) at org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) at org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) at mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) at mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) at mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) at mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) at mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) at mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) at mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) at mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) at mondrian.rolap.RolapResult.(RolapResult.java:279) at mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[ https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2735: -- Issue Type: Sub-task (was: Improvement) Parent: KYLIN-2727 > Introduce an option to make job scheduler consider job priority > --- > > Key: KYLIN-2735 > URL: https://issues.apache.org/jira/browse/KYLIN-2735 > Project: Kylin > Issue Type: Sub-task > Components: Job Engine >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2735) Introduce an option to make job scheduler consider job priority
[ https://issues.apache.org/jira/browse/KYLIN-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhong Yanghong updated KYLIN-2735: -- Summary: Introduce an option to make job scheduler consider job priority (was: Introduce a job scheduler considering job priority) > Introduce an option to make job scheduler consider job priority > --- > > Key: KYLIN-2735 > URL: https://issues.apache.org/jira/browse/KYLIN-2735 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.2.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
