[jira] [Commented] (KYLIN-2796) Remove the legacy "statisticsenabled" codes in FactDistinctColumnsJob
[ https://issues.apache.org/jira/browse/KYLIN-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16239589#comment-16239589 ] Billy Liu commented on KYLIN-2796: -- +1 > Remove the legacy "statisticsenabled" codes in FactDistinctColumnsJob > - > > Key: KYLIN-2796 > URL: https://issues.apache.org/jira/browse/KYLIN-2796 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI > Labels: scope > Fix For: v2.3.0 > > > There are some legacy codes in FactDistinctColumnsJob.java which was for the > compitable with v1.3 and before's Cube, like the checking for whether need > collect cube statistics. In 1.5 and later, this option is alreays be true, > removing the redundant check will make code clean. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2928) PUSH DOWN query cannot use order by function
[ https://issues.apache.org/jira/browse/KYLIN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2928: - Fix Version/s: v2.3.0 > PUSH DOWN query cannot use order by function > > > Key: KYLIN-2928 > URL: https://issues.apache.org/jira/browse/KYLIN-2928 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v2.1.0 >Reporter: Yang Hao >Assignee: Yang Hao > Fix For: v2.3.0 > > > SQL : select "DATE",count(1) from table_1 group by "DATE" order by "DATE" > desc; > Exception:org.apache.calcite.sql.SqlOrderBy cannot be cast to > org.apache.calcite.sql.SqlSelect -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2952) dynamic cube build for time(statTime and endTime)
[
https://issues.apache.org/jira/browse/KYLIN-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212493#comment-16212493
]
Billy Liu commented on KYLIN-2952:
--
The issue is clear. epoch has no timezone itself. It's just a number. The
timezone only works for some GUI display, not the actual data. Plesae use GMT
for API parameter.
> dynamic cube build for time(statTime and endTime)
> -
>
> Key: KYLIN-2952
> URL: https://issues.apache.org/jira/browse/KYLIN-2952
> Project: Kylin
> Issue Type: Improvement
> Components: REST Service
>Affects Versions: v1.6.0
> Environment: linux
>Reporter: wenxue lin
>Assignee: Zhong,Jason
>Priority: Minor
>
> ex => curl -X PUT -u "ADMIN:KYLIN" -H
> "Content-Type:application/json;charset=utf-8" -d
> '{"startTime":150860160,"endTime":150868800,"buildType":"BUILD"}'
> http://host:port/kylin/api/cubes/bi_dispatch_waiting_service_cube/rebuild
> desc :
> rest api param is startTime:150860160(2017-10-22) and
> endTime:150868800(2017-10-23)， but the actual time of building the cube
> is 1 day ahead of schedule (actually 8 hours ahead of schedule)
> =》【2017-10-21~2017-10-22】，But using kylinUI build without question, the
> actual view kylin code, found that is because the kylin on the server side
> source code for configuration of GMT rather than use the timezone of fixed
> GMT + 8, and front-end UI will according to the configuration of the timezone
> is transformed to the time of the page to add GMT + 8 time, then the back-end
> to GMT + 8 time in into GMT time, so the kylinUI cube build time without
> error, and using restAPI build cube time not making timezone 8 hours is not
> accurate time difference problem。
> *for code:*
> kylinProperties.js
> this.getTimeZone = function () {
> if (!this.timezone) {
> this.timezone = this.getProperty("kylin.rest.timezone").trim();
> }
> return this.timezone;
> }
> org.apache.kylin.cube.CubeSegment
> public static String makeSegmentName(long startDate, long endDate, long
> startOffset, long endOffset) {
> if (startOffset != 0 || endOffset != 0) {
> if (startOffset == 0 && (endOffset == 0 || endOffset ==
> Long.MAX_VALUE)) {
> return "FULL_BUILD";
> }
> return startOffset + "_" + endOffset;
> }
> // using time
> SimpleDateFormat dateFormat = new SimpleDateFormat("MMddHHmmss");
> dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
> return dateFormat.format(startDate) + "_" + dateFormat.format(endDate);
> }
> org.apache.kylin.common.util.DateFormat:
> public static FastDateFormat getDateFormat(String datePattern) {
> FastDateFormat r = formatMap.get(datePattern);
> if (r == null) {
> r = FastDateFormat.getInstance(datePattern,
> TimeZone.getTimeZone("GMT")); // NOTE: this must be GMT to calculate epoch
> date correctly
> formatMap.put(datePattern, r);
> }
> return r;
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2952) dynamic cube build for time(statTime and endTime)
[
https://issues.apache.org/jira/browse/KYLIN-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212475#comment-16212475
]
Billy Liu commented on KYLIN-2952:
--
150860160 is GMT: Saturday, October 21, 2017 4:00:00 PM, not 2017-10-22
> dynamic cube build for time(statTime and endTime)
> -
>
> Key: KYLIN-2952
> URL: https://issues.apache.org/jira/browse/KYLIN-2952
> Project: Kylin
> Issue Type: Improvement
> Components: REST Service
>Affects Versions: v1.6.0
> Environment: linux
>Reporter: wenxue lin
>Assignee: Zhong,Jason
>Priority: Minor
>
> ex => curl -X PUT -u "ADMIN:KYLIN" -H
> "Content-Type:application/json;charset=utf-8" -d
> '{"startTime":150860160,"endTime":150868800,"buildType":"BUILD"}'
> http://host:port/kylin/api/cubes/bi_dispatch_waiting_service_cube/rebuild
> desc :
> rest api param is startTime:150860160(2017-10-22) and
> endTime:150868800(2017-10-23)， but the actual time of building the cube
> is 1 day ahead of schedule (actually 8 hours ahead of schedule)
> =》【2017-10-21~2017-10-22】，But using kylinUI build without question, the
> actual view kylin code, found that is because the kylin on the server side
> source code for configuration of GMT rather than use the timezone of fixed
> GMT + 8, and front-end UI will according to the configuration of the timezone
> is transformed to the time of the page to add GMT + 8 time, then the back-end
> to GMT + 8 time in into GMT time, so the kylinUI cube build time without
> error, and using restAPI build cube time not making timezone 8 hours is not
> accurate time difference problem。
> *for code:*
> kylinProperties.js
> this.getTimeZone = function () {
> if (!this.timezone) {
> this.timezone = this.getProperty("kylin.rest.timezone").trim();
> }
> return this.timezone;
> }
> org.apache.kylin.cube.CubeSegment
> public static String makeSegmentName(long startDate, long endDate, long
> startOffset, long endOffset) {
> if (startOffset != 0 || endOffset != 0) {
> if (startOffset == 0 && (endOffset == 0 || endOffset ==
> Long.MAX_VALUE)) {
> return "FULL_BUILD";
> }
> return startOffset + "_" + endOffset;
> }
> // using time
> SimpleDateFormat dateFormat = new SimpleDateFormat("MMddHHmmss");
> dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
> return dateFormat.format(startDate) + "_" + dateFormat.format(endDate);
> }
> org.apache.kylin.common.util.DateFormat:
> public static FastDateFormat getDateFormat(String datePattern) {
> FastDateFormat r = formatMap.get(datePattern);
> if (r == null) {
> r = FastDateFormat.getInstance(datePattern,
> TimeZone.getTimeZone("GMT")); // NOTE: this must be GMT to calculate epoch
> date correctly
> formatMap.put(datePattern, r);
> }
> return r;
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2952) dynamic cube build for time(statTime and endTime)
[
https://issues.apache.org/jira/browse/KYLIN-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212188#comment-16212188
]
Billy Liu commented on KYLIN-2952:
--
The epoch has no timezone info. All API use epoch only. What's your proposal or
question?
> dynamic cube build for time(statTime and endTime)
> -
>
> Key: KYLIN-2952
> URL: https://issues.apache.org/jira/browse/KYLIN-2952
> Project: Kylin
> Issue Type: Improvement
> Components: REST Service
>Affects Versions: v1.6.0
> Environment: linux
>Reporter: wenxue lin
>Assignee: Zhong,Jason
>Priority: Minor
>
> ex => curl -X PUT -u "ADMIN:KYLIN" -H
> "Content-Type:application/json;charset=utf-8" -d
> '{"startTime":150860160,"endTime":150868800,"buildType":"BUILD"}'
> http://host:port/kylin/api/cubes/bi_dispatch_waiting_service_cube/rebuild
> desc :
> rest api param is startTime:150860160(2017-10-22) and
> endTime:150868800(2017-10-23)， but the actual time of building the cube
> is 1 day ahead of schedule (actually 8 hours ahead of schedule)
> =》【2017-10-21~2017-10-22】，But using kylinUI build without question, the
> actual view kylin code, found that is because the kylin on the server side
> source code for configuration of GMT rather than use the timezone of fixed
> GMT + 8, and front-end UI will according to the configuration of the timezone
> is transformed to the time of the page to add GMT + 8 time, then the back-end
> to GMT + 8 time in into GMT time, so the kylinUI cube build time without
> error, and using restAPI build cube time not making timezone 8 hours is not
> accurate time difference problem。
> *for code:*
> kylinProperties.js
> this.getTimeZone = function () {
> if (!this.timezone) {
> this.timezone = this.getProperty("kylin.rest.timezone").trim();
> }
> return this.timezone;
> }
> org.apache.kylin.cube.CubeSegment
> public static String makeSegmentName(long startDate, long endDate, long
> startOffset, long endOffset) {
> if (startOffset != 0 || endOffset != 0) {
> if (startOffset == 0 && (endOffset == 0 || endOffset ==
> Long.MAX_VALUE)) {
> return "FULL_BUILD";
> }
> return startOffset + "_" + endOffset;
> }
> // using time
> SimpleDateFormat dateFormat = new SimpleDateFormat("MMddHHmmss");
> dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
> return dateFormat.format(startDate) + "_" + dateFormat.format(endDate);
> }
> org.apache.kylin.common.util.DateFormat:
> public static FastDateFormat getDateFormat(String datePattern) {
> FastDateFormat r = formatMap.get(datePattern);
> if (r == null) {
> r = FastDateFormat.getInstance(datePattern,
> TimeZone.getTimeZone("GMT")); // NOTE: this must be GMT to calculate epoch
> date correctly
> formatMap.put(datePattern, r);
> }
> return r;
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2943) 关于kylin设置queuename遇到的问题
[ https://issues.apache.org/jira/browse/KYLIN-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207801#comment-16207801 ] Billy Liu commented on KYLIN-2943: -- Please follow http://kylin.apache.org/development/howto_contribute.html Both Github PR and a patch are OK. > 关于kylin设置queuename遇到的问题 > --- > > Key: KYLIN-2943 > URL: https://issues.apache.org/jira/browse/KYLIN-2943 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0 > Environment: CDH5.7 >Reporter: wang20170707 >Assignee: Dong Li > > 在CDH5.7环境下新搭建kylin2.1环境 > 1.需要修改队列名称，具体操作如下 > 在kylin.properties文件配置队列参数如下： > kylin.source.kylin.client=beeline > kylin.engine.mr.config-override.mapreduce.job.queuename=cbasQueue > kylin.source.hive.config-override.mapreduce.job.queuename=cbasQueue > 在kylin_hive_conf.xml里面配 > mapred.job.queue.name > 在 > kylin_job_conf_inmem.xml > kylin_job_conf.xml文件里配 > mapreduce.job.queuename=cbasQueue > 2.修改好配置后重启kylin，在load hive table时，运行的mr任务报错，详情如下： > User: bdcbas1234 > Name: Kylin Hive Column Cardinality Job table=CBAS_DB.S_WEB_APP_PAGE > output=hdfs://zhcbdpII/bdcbasApp/kylin/bdcbasApp-kylin_metadata/cardinality/5ae93815-0f53-412e-bb53-07c194c873d7/CBAS_DB.S_WEB_APP_PAGE > Application Type: MAPREDUCE > Application Tags: > State:FAILED > FinalStatus: FAILED > Started: 星期二 十月 17 16:25:47 +0800 2017 > Elapsed: 8sec > Tracking URL: History > Diagnostics: > Application application_1507516583223_4851 failed 2 times due to AM Container > for appattempt_1507516583223_4851_02 exited with exitCode: 1 due to: > Exception from container-launch. > Container id: container_1507516583223_4851_02_01 > Exit code: 1 > Stack trace: ExitCodeException exitCode=1: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > at org.apache.hadoop.util.Shell.run(Shell.java:455) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) > at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Shell output: main : command provided 1 > main : user is bdcbas1234 > main : requested yarn user is bdcbas1234 > Container exited with a non-zero exit code 1 > .Failing this attempt.. Failing the application. > 3.在BULID CUBE时，在第2步失败： > #1 Step Name: Create Intermediate Flat Hive Table > 执行成功，mr任务使用的queue为配置后的cbasQueue > #2 Step Name: Redistribute Flat Hive Table > 执行失败，mr任务select count(*) from > ...6d_9935_49f0c9600e38(Stage-1)使用的queue为配置前的default > 4.将kylin.properties文件里的参数 > kylin.source.kylin.client修改为cli > 执行load hive table错误与beeline相同 > 5.在BULID CUBE时,在第3步失败 > #1 Step Name: Create Intermediate Flat Hive Table > 执行成功，与beeline相同 > #2 Step Name: Redistribute Flat Hive Table > 执行成功，后台没有运行mr任务，kylin日志打印如下 > Row count is 0, no need to redistribute > #3 Step Name: Extract Fact Table Distinct Columns > 执行失败，日志如下： > User: bdcbas1234 > Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step > Application Type: MAPREDUCE > Application Tags: > State:FAILED > FinalStatus: FAILED > Started: 星期二 十月 17 13:52:40 +0800 2017 > Elapsed: 26sec > Tracking URL: History > Diagnostics: > Application application_1507516583223_4639 failed 2 times due to AM Container > for appattempt_1507516583223_4639_02 exited with exitCode: 1 due to: > Exception from container-launch. > Container id: container_1507516583223_4639_02_01 > Exit code: 1 > Stack trace: ExitCodeException exitCode=1: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > at org.apache.hadoop.util.Shell.run(Shell.java:455) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) > at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at
[jira] [Commented] (KYLIN-2943) 关于kylin设置queuename遇到的问题
[ https://issues.apache.org/jira/browse/KYLIN-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207649#comment-16207649 ] Billy Liu commented on KYLIN-2943: -- Hi [~wang20170707], you have identified the issue. The patch is welcomed. Do you want to have a try? > 关于kylin设置queuename遇到的问题 > --- > > Key: KYLIN-2943 > URL: https://issues.apache.org/jira/browse/KYLIN-2943 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0 > Environment: CDH5.7 >Reporter: wang20170707 >Assignee: Dong Li > > 在CDH5.7环境下新搭建kylin2.1环境 > 1.需要修改队列名称，具体操作如下 > 在kylin.properties文件配置队列参数如下： > kylin.source.kylin.client=beeline > kylin.engine.mr.config-override.mapreduce.job.queuename=cbasQueue > kylin.source.hive.config-override.mapreduce.job.queuename=cbasQueue > 在kylin_hive_conf.xml里面配 > mapred.job.queue.name > 在 > kylin_job_conf_inmem.xml > kylin_job_conf.xml文件里配 > mapreduce.job.queuename=cbasQueue > 2.修改好配置后重启kylin，在load hive table时，运行的mr任务报错，详情如下： > User: bdcbas1234 > Name: Kylin Hive Column Cardinality Job table=CBAS_DB.S_WEB_APP_PAGE > output=hdfs://zhcbdpII/bdcbasApp/kylin/bdcbasApp-kylin_metadata/cardinality/5ae93815-0f53-412e-bb53-07c194c873d7/CBAS_DB.S_WEB_APP_PAGE > Application Type: MAPREDUCE > Application Tags: > State:FAILED > FinalStatus: FAILED > Started: 星期二 十月 17 16:25:47 +0800 2017 > Elapsed: 8sec > Tracking URL: History > Diagnostics: > Application application_1507516583223_4851 failed 2 times due to AM Container > for appattempt_1507516583223_4851_02 exited with exitCode: 1 due to: > Exception from container-launch. > Container id: container_1507516583223_4851_02_01 > Exit code: 1 > Stack trace: ExitCodeException exitCode=1: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > at org.apache.hadoop.util.Shell.run(Shell.java:455) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) > at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Shell output: main : command provided 1 > main : user is bdcbas1234 > main : requested yarn user is bdcbas1234 > Container exited with a non-zero exit code 1 > .Failing this attempt.. Failing the application. > 3.在BULID CUBE时，在第2步失败： > #1 Step Name: Create Intermediate Flat Hive Table > 执行成功，mr任务使用的queue为配置后的cbasQueue > #2 Step Name: Redistribute Flat Hive Table > 执行失败，mr任务select count(*) from > ...6d_9935_49f0c9600e38(Stage-1)使用的queue为配置前的default > 4.将kylin.properties文件里的参数 > kylin.source.kylin.client修改为cli > 执行load hive table错误与beeline相同 > 5.在BULID CUBE时,在第3步失败 > #1 Step Name: Create Intermediate Flat Hive Table > 执行成功，与beeline相同 > #2 Step Name: Redistribute Flat Hive Table > 执行成功，后台没有运行mr任务，kylin日志打印如下 > Row count is 0, no need to redistribute > #3 Step Name: Extract Fact Table Distinct Columns > 执行失败，日志如下： > User: bdcbas1234 > Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step > Application Type: MAPREDUCE > Application Tags: > State:FAILED > FinalStatus: FAILED > Started: 星期二 十月 17 13:52:40 +0800 2017 > Elapsed: 26sec > Tracking URL: History > Diagnostics: > Application application_1507516583223_4639 failed 2 times due to AM Container > for appattempt_1507516583223_4639_02 exited with exitCode: 1 due to: > Exception from container-launch. > Container id: container_1507516583223_4639_02_01 > Exit code: 1 > Stack trace: ExitCodeException exitCode=1: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) > at org.apache.hadoop.util.Shell.run(Shell.java:455) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) > at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at
[jira] [Commented] (KYLIN-2937) 非分区cube的中间数据会累积
[ https://issues.apache.org/jira/browse/KYLIN-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205141#comment-16205141 ] Billy Liu commented on KYLIN-2937: -- Please check the cleanup job log, it should tell why some resources are kept from removing. > 非分区cube的中间数据会累积 > --- > > Key: KYLIN-2937 > URL: https://issues.apache.org/jira/browse/KYLIN-2937 > Project: Kylin > Issue Type: Bug >Reporter: zhengzfand > > 非分区的cube构建之后,中间数据不会被清理. > 存放在hdfs上的字典文件,会一直累积.cube构建时会加载所有这些累积字典文件, > 有可能导致内存溢出(如果字典文件够大够多的话). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2927) Merge Cuboid Dictionary ERROR
[ https://issues.apache.org/jira/browse/KYLIN-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205140#comment-16205140 ] Billy Liu commented on KYLIN-2927: -- First, the code change happens in core-dictionary, it works in kylin-job actually, not kylin-tool. Second, the proposal workaround is try to build Kylin package from the branch 2.2.X. > Merge Cuboid Dictionary ERROR > - > > Key: KYLIN-2927 > URL: https://issues.apache.org/jira/browse/KYLIN-2927 > Project: Kylin > Issue Type: Bug >Reporter: songxiangjun > > when i merge the segment encount error, how can i solve it. Log as follows: > 2017-10-11 15:24:56,139 ERROR [pool-9-thread-10] > threadpool.DefaultScheduler:145 : ExecuteException > job:7508dfa0-5a89-4c3c-8685-701226628207 > org.apache.kylin.job.exception.ExecuteException: > org.apache.kylin.job.exception.ExecuteException: > java.lang.IllegalStateException: Invalid input data. Unordered data cannot be > split into multi trees > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:135) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:141) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.kylin.job.exception.ExecuteException: > java.lang.IllegalStateException: Invalid input data. Unordered data cannot be > split into multi trees > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:135) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:65) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) > ... 4 more > Caused by: java.lang.IllegalStateException: Invalid input data. Unordered > data cannot be split into multi trees > at > org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:92) > at > org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:78) > at > org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.addValue(DictionaryGenerator.java:261) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:79) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:64) > at > org.apache.kylin.dict.DictionaryGenerator.mergeDictionaries(DictionaryGenerator.java:104) > at > org.apache.kylin.dict.DictionaryManager.mergeDictionary(DictionaryManager.java:275) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.mergeDictionaries(MergeDictionaryStep.java:146) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.makeDictForNewSegment(MergeDictionaryStep.java:136) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.doWork(MergeDictionaryStep.java:68) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) > ... 6 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2937) 非分区cube的中间数据会累积
[ https://issues.apache.org/jira/browse/KYLIN-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203643#comment-16203643 ] Billy Liu commented on KYLIN-2937: -- As the log said, "Identical dictionary content, reuse existing dictionary" In your case, no new dictionary is building. Have you run the StorageCleanupJob? > 非分区cube的中间数据会累积 > --- > > Key: KYLIN-2937 > URL: https://issues.apache.org/jira/browse/KYLIN-2937 > Project: Kylin > Issue Type: Bug >Reporter: zhengzfand > > 非分区的cube构建之后,中间数据不会被清理. > 存放在hdfs上的字典文件,会一直累积.cube构建时会加载所有这些累积字典文件, > 有可能导致内存溢出(如果字典文件够大够多的话). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2938) Project Update ACL issue
[ https://issues.apache.org/jira/browse/KYLIN-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203311#comment-16203311 ] Billy Liu commented on KYLIN-2938: -- Patch is welcomed. > Project Update ACL issue > > > Key: KYLIN-2938 > URL: https://issues.apache.org/jira/browse/KYLIN-2938 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v2.1.0 >Reporter: Pan, Julian >Assignee: Zhong,Jason > > I noticed there different security between updateProject and renameProject in > ProjectService. > @PreAuthorize(Constant.ACCESS_HAS_ROLE_ADMIN + " or > hasPermission(#currentProject, 'ADMINISTRATION') or > hasPermission(#currentProject, 'MANAGEMENT')") > updateProject > @PreAuthorize(Constant.ACCESS_HAS_ROLE_ADMIN + " or > hasPermission(#currentProject, 'ADMINISTRATION')") > renameProject > Even updateProject will call renameProject. Which will throws exception if > user is MANAGEMENT role. > The renameProject should be same security check with updateProject. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2937) 非分区cube的中间数据会累积
[ https://issues.apache.org/jira/browse/KYLIN-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203310#comment-16203310 ] Billy Liu commented on KYLIN-2937: -- [~zfzheng], thank you for reporting this issue. Could you update the Kylin version, and upload some snapshots evidence? > 非分区cube的中间数据会累积 > --- > > Key: KYLIN-2937 > URL: https://issues.apache.org/jira/browse/KYLIN-2937 > Project: Kylin > Issue Type: Bug >Reporter: zhengzfand > > 非分区的cube构建之后,中间数据不会被清理. > 存放在hdfs上的字典文件,会一直累积.cube构建时会加载所有这些累积字典文件, > 有可能导致内存溢出(如果字典文件够大够多的话). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2938) Project Update ACL issue
[ https://issues.apache.org/jira/browse/KYLIN-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203305#comment-16203305 ] Billy Liu commented on KYLIN-2938: -- Agree. They should be the same permission. > Project Update ACL issue > > > Key: KYLIN-2938 > URL: https://issues.apache.org/jira/browse/KYLIN-2938 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v2.1.0 >Reporter: Pan, Julian >Assignee: Zhong,Jason > > I noticed there different security between updateProject and renameProject in > ProjectService. > @PreAuthorize(Constant.ACCESS_HAS_ROLE_ADMIN + " or > hasPermission(#currentProject, 'ADMINISTRATION') or > hasPermission(#currentProject, 'MANAGEMENT')") > updateProject > @PreAuthorize(Constant.ACCESS_HAS_ROLE_ADMIN + " or > hasPermission(#currentProject, 'ADMINISTRATION')") > renameProject > Even updateProject will call renameProject. Which will throws exception if > user is MANAGEMENT role. > The renameProject should be same security check with updateProject. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2794) MultipleDictionaryValueEnumerator should output values in sorted order
[
https://issues.apache.org/jira/browse/KYLIN-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2794:
-
Fix Version/s: (was: v2.3.0)
v2.2.0
> MultipleDictionaryValueEnumerator should output values in sorted order
> --
>
> Key: KYLIN-2794
> URL: https://issues.apache.org/jira/browse/KYLIN-2794
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.0.0
> Environment: hadoop hadoop-2.6.0-cdh5.8.2 hive 2.1 hbase 0.98
>Reporter: 翟玉勇
>Assignee: Yifei Wu
>Priority: Critical
> Labels: scope
> Fix For: v2.2.0
>
>
> Dictionary exception during merge of segments.
> {code}
> 2017-08-18 14:17:48,828 ERROR [pool-11-thread-1]
> threadpool.DistributedScheduler:188 : ExecuteException
> job:8d031b5f-2d3f-445f-a62b-7bc560d919ea in server: **
> org.apache.kylin.job.exception.ExecuteException:
> org.apache.kylin.job.exception.ExecuteException:
> java.lang.IllegalStateException: Invalid input data. Unordered data cannot be
> split into multi trees
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:134)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:185)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kylin.job.exception.ExecuteException:
> java.lang.IllegalStateException: Invalid input data. Unordered data cannot be
> split into multi trees
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:134)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
> ... 4 more
> Caused by: java.lang.IllegalStateException: Invalid input data. Unordered
> data cannot be split into multi trees
> at
> org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:92)
> at
> org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:78)
> at
> org.apache.kylin.dict.DictionaryGenerator$StringTrieDictForestBuilder.addValue(DictionaryGenerator.java:212)
> at
> org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:79)
> at
> org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:64)
> at
> org.apache.kylin.dict.DictionaryGenerator.mergeDictionaries(DictionaryGenerator.java:104)
> at
> org.apache.kylin.dict.DictionaryManager.mergeDictionary(DictionaryManager.java:267)
> at
> org.apache.kylin.engine.mr.steps.MergeDictionaryStep.mergeDictionaries(MergeDictionaryStep.java:146)
> at
> org.apache.kylin.engine.mr.steps.MergeDictionaryStep.makeDictForNewSegment(MergeDictionaryStep.java:136)
> at
> org.apache.kylin.engine.mr.steps.MergeDictionaryStep.doWork(MergeDictionaryStep.java:68)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
> ... 6 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2931) Update correct cardinality for the modified table
[ https://issues.apache.org/jira/browse/KYLIN-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201547#comment-16201547 ] Billy Liu commented on KYLIN-2931: -- yes. [~peng.jianhua] > Update correct cardinality for the modified table > - > > Key: KYLIN-2931 > URL: https://issues.apache.org/jira/browse/KYLIN-2931 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.0.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Attachments: > 0001-KYLIN-2931-Update-correct-cardinality-for-the-modifi.patch, 01.PNG, > 02.PNG, 03.PNG, 04.PNG, 05.PNG, 06.PNG, 07.PNG > > > 1,Add project_1 and project_2,each project has the table kylin_sales_copy; > 2,Do the following steps: > a,use 'create table kylin_sales_copy as select * from kylin_sales;' to copy > table kylin_sales; > b,load table kylin_sales_copy and calculate column cardinality in project_1 > and project_2,please refer to 01.png; > c,use 'drop table kylin_sales_copy;' to drop table kylin_sales_copy; > d,use 'create table kylin_sales_copy as select > TRANS_ID,PART_DT,LSTG_FORMAT_NAME,LEAF_CATEG_ID,LSTG_SITE_ID,SLR_SEGMENT_CD,PRICE > from kylin_sales where price < 20;' to create the new kylin_sales_copy table; > Now the structure of kylin_sales_copy has changed, the same as the > cardinality. > 3,Reload table kylin_sales_copy in project_1 or project_2,we found that the > cardinality is not change,please refer to 02.png. > However,I found that the new cardinality was calculated but not update on the > web by debug the code,please refer to 03.png,04.png.It should update correct > cardinality like 05.png. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2931) Update correct cardinality for the modified table
[ https://issues.apache.org/jira/browse/KYLIN-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201528#comment-16201528 ] Billy Liu commented on KYLIN-2931: -- [~peng.jianhua] could you dump the metadata, and check the table_ext directory. That's what [~wormholer] looking for. > Update correct cardinality for the modified table > - > > Key: KYLIN-2931 > URL: https://issues.apache.org/jira/browse/KYLIN-2931 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.0.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Attachments: > 0001-KYLIN-2931-Update-correct-cardinality-for-the-modifi.patch, 01.PNG, > 02.PNG, 03.PNG, 04.PNG, 05.PNG, 06.PNG, 07.PNG > > > 1,Add project_1 and project_2,each project has the table kylin_sales_copy; > 2,Do the following steps: > a,use 'create table kylin_sales_copy as select * from kylin_sales;' to copy > table kylin_sales; > b,load table kylin_sales_copy and calculate column cardinality in project_1 > and project_2,please refer to 01.png; > c,use 'drop table kylin_sales_copy;' to drop table kylin_sales_copy; > d,use 'create table kylin_sales_copy as select > TRANS_ID,PART_DT,LSTG_FORMAT_NAME,LEAF_CATEG_ID,LSTG_SITE_ID,SLR_SEGMENT_CD,PRICE > from kylin_sales where price < 20;' to create the new kylin_sales_copy table; > Now the structure of kylin_sales_copy has changed, the same as the > cardinality. > 3,Reload table kylin_sales_copy in project_1 or project_2,we found that the > cardinality is not change,please refer to 02.png. > However,I found that the new cardinality was calculated but not update on the > web by debug the code,please refer to 03.png,04.png.It should update correct > cardinality like 05.png. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2600) Incorrectly set the range start when filtering by the minimum value
[
https://issues.apache.org/jira/browse/KYLIN-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201313#comment-16201313
]
Billy Liu commented on KYLIN-2600:
--
Hi [~yaho], could you double commit this patch to the 2.2 branch also?
> Incorrectly set the range start when filtering by the minimum value
> ---
>
> Key: KYLIN-2600
> URL: https://issues.apache.org/jira/browse/KYLIN-2600
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Fix For: v2.2.0
>
>
> Before defining a range of a scan, the range start may be not correct in the
> following case:
> {code}
> OR [
> AND [
> a <= 3,
> b = 2
> ],
> AND [
> a = 0, (note that 0 is the minimum of a)
> b = 1
> ]
> ]
> {code}
> In this case, kylin will generate two ranges:
> {code}
> [null,2] - [3, 2]
> [0,1] - [0,1]
> {code}
> There's a rule when merging ranges. If the range start is null, it will be
> ordered before others whose range start is not null. Then the merged range of
> these two ranges will be
> \[null,2\] - \[3, 2\].
> Replacing null with 0, the range sent to coprocessor will be
> \[0,2\] - \[3, 2\].
> However, it should be
> \[0,1\] - \[3, 2\]
> and the rule should be refined.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2927) Merge Cuboid Dictionary ERROR
[ https://issues.apache.org/jira/browse/KYLIN-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1628#comment-1628 ] Billy Liu commented on KYLIN-2927: -- Should have been fixed at https://issues.apache.org/jira/browse/KYLIN-2794 > Merge Cuboid Dictionary ERROR > - > > Key: KYLIN-2927 > URL: https://issues.apache.org/jira/browse/KYLIN-2927 > Project: Kylin > Issue Type: Bug >Reporter: songxiangjun > > when i merge the segment encount error, how can i solve it. Log as follows: > 2017-10-11 15:24:56,139 ERROR [pool-9-thread-10] > threadpool.DefaultScheduler:145 : ExecuteException > job:7508dfa0-5a89-4c3c-8685-701226628207 > org.apache.kylin.job.exception.ExecuteException: > org.apache.kylin.job.exception.ExecuteException: > java.lang.IllegalStateException: Invalid input data. Unordered data cannot be > split into multi trees > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:135) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:141) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.kylin.job.exception.ExecuteException: > java.lang.IllegalStateException: Invalid input data. Unordered data cannot be > split into multi trees > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:135) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:65) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) > ... 4 more > Caused by: java.lang.IllegalStateException: Invalid input data. Unordered > data cannot be split into multi trees > at > org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:92) > at > org.apache.kylin.dict.TrieDictionaryForestBuilder.addValue(TrieDictionaryForestBuilder.java:78) > at > org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.addValue(DictionaryGenerator.java:261) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:79) > at > org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:64) > at > org.apache.kylin.dict.DictionaryGenerator.mergeDictionaries(DictionaryGenerator.java:104) > at > org.apache.kylin.dict.DictionaryManager.mergeDictionary(DictionaryManager.java:275) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.mergeDictionaries(MergeDictionaryStep.java:146) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.makeDictForNewSegment(MergeDictionaryStep.java:136) > at > org.apache.kylin.engine.mr.steps.MergeDictionaryStep.doWork(MergeDictionaryStep.java:68) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125) > ... 6 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2928) PUSH DOWN query cannot use order by function
[ https://issues.apache.org/jira/browse/KYLIN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1626#comment-1626 ] Billy Liu commented on KYLIN-2928: -- Which pushdown engine you are using? > PUSH DOWN query cannot use order by function > > > Key: KYLIN-2928 > URL: https://issues.apache.org/jira/browse/KYLIN-2928 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Yang Hao >Assignee: liyang > > SQL : select "DATE",count(1) from table_1 group by "DATE" order by "DATE" > desc; > Exception:org.apache.calcite.sql.SqlOrderBy cannot be cast to > org.apache.calcite.sql.SqlSelect -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu resolved KYLIN-2891.
--
Resolution: Fixed
Fix Version/s: v2.2.0
https://github.com/apache/kylin/commit/4cc32eca0e6e3ae8e3c2861a6a5e4dc774c896d9
LGTM. Patch merged. Thank you [~peng.jianhua]
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Fix For: v2.2.0
>
> Attachments:
> 0001-KYLIN-2891-Tomcat-Security-Vulnerability-Alert.-The-.patch
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199677#comment-16199677
]
Billy Liu commented on KYLIN-2891:
--
Hi [~peng.jianhua], please take care the package script also. The tomcat md5
needs to be updated.
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Attachments:
> 0001-KYLIN-2891-Tomcat-Security-Vulnerability-Alert.-The-.patch
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2891:
-
Comment: was deleted
(was: https://git1-us-west.apache.org/repos/asf?p=kylin.git;a=commit;h=2a47f445)
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: Billy Liu
> Labels: patch
> Fix For: v2.2.0
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reopened KYLIN-2891:
--
Assignee: peng.jianhua (was: Billy Liu)
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Fix For: v2.2.0
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2891:
-
Fix Version/s: (was: v2.2.0)
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2891) Tomcat Security Vulnerability Alert. The version of the tomcat for kylin should upgrade to 7.0.82.
[
https://issues.apache.org/jira/browse/KYLIN-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu resolved KYLIN-2891.
--
Resolution: Fixed
Assignee: Billy Liu (was: peng.jianhua)
Fix Version/s: v2.2.0
https://git1-us-west.apache.org/repos/asf?p=kylin.git;a=commit;h=2a47f445
> Tomcat Security Vulnerability Alert. The version of the tomcat for kylin
> should upgrade to 7.0.82.
> --
>
> Key: KYLIN-2891
> URL: https://issues.apache.org/jira/browse/KYLIN-2891
> Project: Kylin
> Issue Type: Bug
> Components: Website
>Affects Versions: v2.0.0, v2.1.0
>Reporter: peng.jianhua
>Assignee: Billy Liu
> Labels: patch
> Fix For: v2.2.0
>
>
> 【Security Vulnerability Alert】Tomcat Information leakage and remote code
> execution vulnerabilities.
> CVE ID:
> {code}
> CVE-2017-12617
> {code}
> Description
> {code}
> When running with HTTP PUTs enabled (e.g. via setting the readonly
> initialisation parameter of the Default servlet to false) it was possible to
> upload a JSP file to the server via a specially crafted request. This JSP
> could then be requested and any code it contained would be executed by the
> server.
> {code}
> Scope
> {code}
> Affects: 7.0.0 to 7.0.81
> {code}
> Solution
> {code}
> The official release of the Apache Tomcat 7.0.82 version has fixed the
> vulnerability and recommends upgrading to the 7.0.82 version.
> {code}
> Reference
> {code}
> https://tomcat.apache.org/security-7.html#Fixed_in_Apache_Tomcat_7.0.82
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2812) Save to wrong database when loading Kafka Topic
[ https://issues.apache.org/jira/browse/KYLIN-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185944#comment-16185944 ] Billy Liu commented on KYLIN-2812: -- Thanks Shaofeng‘s review. Now has renamed to timestampField. The change is back-compatible. All properties including tsColName will be included as before. > Save to wrong database when loading Kafka Topic > > > Key: KYLIN-2812 > URL: https://issues.apache.org/jira/browse/KYLIN-2812 > Project: Kylin > Issue Type: Bug >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.2.0 > > > When loading Kafka Topic, user could select the destination database name. > Currently, all topics are saved into DEFAULT database, which is not expected. > Here is a bug during reading create table parameter. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2775) Streaming Cube Sample
[ https://issues.apache.org/jira/browse/KYLIN-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178059#comment-16178059 ] Billy Liu edited comment on KYLIN-2775 at 9/24/17 2:33 AM: --- Commit at https://github.com/apache/kylin/commit/5c3b40be166d837d15df677ee23b9a56f974b799 The sample.sh will generate the following sample objects into learn_kylin project: Table: DEFAULT.KYLIN_STREAMING_TABLE Model: kylin_streaming_model Cube: kylin_streaming_cube The sample-streaming.sh script will generate Kafka Topic: kylin_streaming_topic. It will also send random 100 messages to this topic per second. This script will assume the Kafka broker is located at localhost:9092 To use this sample: 1. Setup Kafka broker at localhost:9092, if other location, the related sample script should be updated also. 2. Export KAFKA_HOME first, and start Kylin. 3. User needs execute sample.sh first and check if the table/model/cube has loaded into Kylin. 4. Run sample-streaming.sh to send messages into Kafka Topic. 5. Trigger Cube build on kylin_streaming_cube. was (Author: yimingliu): Commit at https://github.com/apache/kylin/commit/5c3b40be166d837d15df677ee23b9a56f974b799 The sample.sh will generate the following sample objects into learn_kylin project: Table: DEFAULT.KYLIN_STREAMING_TABLE Model: kylin_streaming_model Cube: kylin_streaming_cube The sample-streaming.sh script will generate Kafka Topic: kylin_streaming_topic. It will also send random 100 messages to this topic per second. This script will assume the Kafka broker is located at localhost:9092 > Streaming Cube Sample > - > > Key: KYLIN-2775 > URL: https://issues.apache.org/jira/browse/KYLIN-2775 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.2.0 > > > The sample.sh will generate sample table/model/cube for Hive-based data > source. There is no easy way to generate sample table/model/cube for > Kafka-based streaming cube. > In this issue, the easy to use streaming sample will be provided. > Will suppose user has Kafka installed already. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2775) Streaming Cube Sample
[ https://issues.apache.org/jira/browse/KYLIN-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2775. -- Resolution: Fixed Fix Version/s: v2.2.0 Commit at https://github.com/apache/kylin/commit/5c3b40be166d837d15df677ee23b9a56f974b799 The sample.sh will generate the following sample objects into learn_kylin project: Table: DEFAULT.KYLIN_STREAMING_TABLE Model: kylin_streaming_model Cube: kylin_streaming_cube The sample-streaming.sh script will generate Kafka Topic: kylin_streaming_topic. It will also send random 100 messages to this topic per second. This script will assume the Kafka broker is located at localhost:9092 > Streaming Cube Sample > - > > Key: KYLIN-2775 > URL: https://issues.apache.org/jira/browse/KYLIN-2775 > Project: Kylin > Issue Type: New Feature > Components: General >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.2.0 > > > The sample.sh will generate sample table/model/cube for Hive-based data > source. There is no easy way to generate sample table/model/cube for > Kafka-based streaming cube. > In this issue, the easy to use streaming sample will be provided. > Will suppose user has Kafka installed already. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2879) Upgrade Spring & Spring Security to fix potential vulnerability
[ https://issues.apache.org/jira/browse/KYLIN-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2879. -- Resolution: Fixed Fix Version/s: v2.2.0 Commit at https://github.com/apache/kylin/commit/13293bad2ce503a35339a5c4ec5bc532684efb48 In this commit, Spring framework is upgraded to 4.3.10 Spring security framework is upgraded to 4.2.3 commons-email is upgraded to 1.5 commons-upload is upgraded to 1.3.3 All potential vulnerablies are resovled. > Upgrade Spring & Spring Security to fix potential vulnerability > --- > > Key: KYLIN-2879 > URL: https://issues.apache.org/jira/browse/KYLIN-2879 > Project: Kylin > Issue Type: Improvement >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Critical > Fix For: v2.2.0 > > > After running against VersionEye, the system shows that Kylin has "14 known > security vulnerabilities. ". They are from commons-fileupload, commons-email, > xercesImpl, spring-webmvc, spring jdbc, spring aop, spring-context-support, > spring-test, spring-security-core, tomcat-catalina, spring-core libraries. > Upgrade to newer version will fix the vulnerabilities. > Following is the detail report: > commons-fileupload : 1.3.1 > 2016-3092 > Apache Commons Fileupload: Denial of Service > https://bugzilla.redhat.com/show_bug.cgi?id=1349475 > http://mail-archives.us.apache.org/mod_mbox/www-announce/201606.mbox/%3c6223ece6-2b41-ef4f-22f9-d3481e492...@apache.org%3E > http://tomcat.apache.org/security.html > http://svn.apache.org/viewvc/commons/proper/fileupload/trunk/RELEASE-NOTES.txt?r1=1745717=1749637_format=h > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092 > Affected versions: <=1.3.1,1.3 && <=1.2.2,1.2 > Mute this security issue > CVE-2016-3092 > CVE-2016-3092 > https://h20566.www2.hpe.com/portal/site/hpsc/public/kb/docDisplay?docId=emr_na-c05324759 > Affected versions: 1.3.1 > Mute this security issue > 2016-131 > Apache Commons FileUpload Deserialization Gadget > https://www.tenable.com/security/research/tra-2016-12 > https://issues.apache.org/jira/browse/FILEUPLOAD-279 > https://nvd.nist.gov/vuln/detail/CVE-2016-131 > Affected versions: <=1.3.2 > Mute this security issue > commons-email : 1.4 > 2017-9801 > SMTP header injection vulnerabilty > https://commons.apache.org/proper/commons-email/security-reports.html > https://nvd.nist.gov/vuln/detail/CVE-2017-9801 > Affected versions: <=1.4 > Mute this security issue > xercesImpl : 2.11.0 > 2013-4002 > Apache Xerces: XMLScanner resource exhaustion > https://bugzilla.redhat.com/CVE-2013-4002 > http://svn.apache.org/viewvc?view=revision=1499506 > Affected versions: <=2.11.0 > Mute this security issue > spring-webmvc : 4.2.8.RELEASE > CVE-2016-9878 > CVE-2016-9878 > https://pivotal.io/security/cve-2016-9878 > Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, > 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, > 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, > 3.2.11, 3.2.10, 3.2.4, 3.2.17 > Mute this security issue > spring-jdbc : 4.2.8.RELEASE > CVE-2016-9878 > CVE-2016-9878 > https://pivotal.io/security/cve-2016-9878 > Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, > 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, > 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, > 3.2.11, 3.2.10, 3.2.4, 3.2.17 > Mute this security issue > spring-aop : 4.2.8.RELEASE > CVE-2016-9878 > CVE-2016-9878 > https://pivotal.io/security/cve-2016-9878 > Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, > 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, > 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, > 3.2.11, 3.2.10, 3.2.4, 3.2.17 > Mute this security issue > spring-context-support : 4.2.8.RELEASE > CVE-2016-9878 > CVE-2016-9878 > https://pivotal.io/security/cve-2016-9878 > Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, > 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, > 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, > 3.2.11, 3.2.10, 3.2.4, 3.2.17 > Mute this security issue > spring-test : 4.2.8.RELEASE > CVE-2016-9878 > CVE-2016-9878 > https://pivotal.io/security/cve-2016-9878 > Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, > 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, > 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, > 3.2.11, 3.2.10, 3.2.4, 3.2.17 > Mute this security issue > spring-security-core : 4.0.4.RELEASE > 2016-5007 > Spring Security / MVC Path Matching Inconsistency >
[jira] [Created] (KYLIN-2879) Upgrade Spring & Spring Security to fix potential vulnerability
Billy Liu created KYLIN-2879: Summary: Upgrade Spring & Spring Security to fix potential vulnerability Key: KYLIN-2879 URL: https://issues.apache.org/jira/browse/KYLIN-2879 Project: Kylin Issue Type: Improvement Reporter: Billy Liu Assignee: Billy Liu Priority: Critical After running against VersionEye, the system shows that Kylin has "14 known security vulnerabilities. ". They are from commons-fileupload, commons-email, xercesImpl, spring-webmvc, spring jdbc, spring aop, spring-context-support, spring-test, spring-security-core, tomcat-catalina, spring-core libraries. Upgrade to newer version will fix the vulnerabilities. Following is the detail report: commons-fileupload : 1.3.1 2016-3092 Apache Commons Fileupload: Denial of Service https://bugzilla.redhat.com/show_bug.cgi?id=1349475 http://mail-archives.us.apache.org/mod_mbox/www-announce/201606.mbox/%3c6223ece6-2b41-ef4f-22f9-d3481e492...@apache.org%3E http://tomcat.apache.org/security.html http://svn.apache.org/viewvc/commons/proper/fileupload/trunk/RELEASE-NOTES.txt?r1=1745717=1749637_format=h https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092 Affected versions: <=1.3.1,1.3 && <=1.2.2,1.2 Mute this security issue CVE-2016-3092 CVE-2016-3092 https://h20566.www2.hpe.com/portal/site/hpsc/public/kb/docDisplay?docId=emr_na-c05324759 Affected versions: 1.3.1 Mute this security issue 2016-131 Apache Commons FileUpload Deserialization Gadget https://www.tenable.com/security/research/tra-2016-12 https://issues.apache.org/jira/browse/FILEUPLOAD-279 https://nvd.nist.gov/vuln/detail/CVE-2016-131 Affected versions: <=1.3.2 Mute this security issue commons-email : 1.4 2017-9801 SMTP header injection vulnerabilty https://commons.apache.org/proper/commons-email/security-reports.html https://nvd.nist.gov/vuln/detail/CVE-2017-9801 Affected versions: <=1.4 Mute this security issue xercesImpl : 2.11.0 2013-4002 Apache Xerces: XMLScanner resource exhaustion https://bugzilla.redhat.com/CVE-2013-4002 http://svn.apache.org/viewvc?view=revision=1499506 Affected versions: <=2.11.0 Mute this security issue spring-webmvc : 4.2.8.RELEASE CVE-2016-9878 CVE-2016-9878 https://pivotal.io/security/cve-2016-9878 Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, 3.2.11, 3.2.10, 3.2.4, 3.2.17 Mute this security issue spring-jdbc : 4.2.8.RELEASE CVE-2016-9878 CVE-2016-9878 https://pivotal.io/security/cve-2016-9878 Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, 3.2.11, 3.2.10, 3.2.4, 3.2.17 Mute this security issue spring-aop : 4.2.8.RELEASE CVE-2016-9878 CVE-2016-9878 https://pivotal.io/security/cve-2016-9878 Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, 3.2.11, 3.2.10, 3.2.4, 3.2.17 Mute this security issue spring-context-support : 4.2.8.RELEASE CVE-2016-9878 CVE-2016-9878 https://pivotal.io/security/cve-2016-9878 Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, 3.2.11, 3.2.10, 3.2.4, 3.2.17 Mute this security issue spring-test : 4.2.8.RELEASE CVE-2016-9878 CVE-2016-9878 https://pivotal.io/security/cve-2016-9878 Affected versions: 4.3.2, 4.2.3, 4.3.0, 4.2.1, 3.2.8, 4.2.0, 4.3.4, 4.2.5, 4.3.1, 4.2.2, 3.2.9, 4.3.3, 4.2.4, 3.2.7, 3.2.0, 4.2.6, 3.2.16, 3.2.5, 3.2.13, 3.2.6, 4.2.8, 3.2.3, 3.2.15, 3.2.12, 3.2.1, 4.2.7, 3.2.2, 3.2.14, 3.2.11, 3.2.10, 3.2.4, 3.2.17 Mute this security issue spring-security-core : 4.0.4.RELEASE 2016-5007 Spring Security / MVC Path Matching Inconsistency https://pivotal.io/security/cve-2016-5007 Affected versions: <=4.1.0.RELEASE Mute this security issue tomcat-catalina : 7.0.69 2016-3092 Apache Commons Fileupload: Denial of Service https://bugzilla.redhat.com/show_bug.cgi?id=1349475 http://mail-archives.us.apache.org/mod_mbox/www-announce/201606.mbox/%3c6223ece6-2b41-ef4f-22f9-d3481e492...@apache.org%3E http://tomcat.apache.org/security.html http://svn.apache.org/viewvc/commons/proper/fileupload/trunk/RELEASE-NOTES.txt?r1=1745717=1749637_format=h https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-3092 Affected versions: <=9.0.0.M7,9 && <=8.5.2,8.5 && <=8.0.35,8.0 && <=7.0.69,7 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2875) Cube e-mail notification Validation
[ https://issues.apache.org/jira/browse/KYLIN-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169857#comment-16169857 ] Billy Liu commented on KYLIN-2875: -- Thanks [~julianpan] for reporting. It seems we should add check from front-end. > Cube e-mail notification Validation > --- > > Key: KYLIN-2875 > URL: https://issues.apache.org/jira/browse/KYLIN-2875 > Project: Kylin > Issue Type: Improvement > Components: REST Service, Web >Reporter: Pan, Julian >Assignee: Pan, Julian > > Validation e-mail notification list avoid EmailException: > 2017-03-28 20:01:56,453 INFO [pool-10-thread-8] > execution.AbstractExecutable:250 : notify > list:[us...@apache.com;us...@apache.com;, dl-user-gr...@apache.com] > 2017-03-28 20:01:56,453 ERROR [pool-10-thread-8] util.MailService:112 : > Illegal semicolon, not in group > org.apache.commons.mail.EmailException: Illegal semicolon, not in group > at org.apache.commons.mail.Email.createInternetAddress(Email.java:541) > at org.apache.commons.mail.Email.addTo(Email.java:653) > at org.apache.commons.mail.Email.addTo(Email.java:636) > at org.apache.commons.mail.Email.addTo(Email.java:618) > at org.apache.kylin.common.util.MailService.sendMail(MailService.java:97) > at org.apache.kylin.common.util.MailService.sendMail(MailService.java:70) > at > org.apache.kylin.job.execution.AbstractExecutable.notifyUserStatusChange(AbstractExecutable.java:251) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.onStatusChange(DefaultChainedExecutable.java:105) > at org.apache.kylin.engine.mr.CubingJob.onStatusChange(CubingJob.java:296) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.onExecuteFinished(DefaultChainedExecutable.java:89) > at org.apache.kylin.engine.mr.CubingJob.onExecuteFinished(CubingJob.java:292) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:135) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: javax.mail.internet.AddressException: Illegal semicolon, not in > group in string > ``cais...@ebay.com;kalagupack...@ebay.com;cb...@ebay.com;cb...@ebay.com;'' at > position 16 > at javax.mail.internet.InternetAddress.parse(InternetAddress.java:780) > at javax.mail.internet.InternetAddress.parse(InternetAddress.java:555) > at javax.mail.internet.InternetAddress.(InternetAddress.java:91) > at org.apache.commons.mail.Email.createInternetAddress(Email.java:514) > ... 15 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2815) Empty results with prepareStatement but OK with KylinStatement
[
https://issues.apache.org/jira/browse/KYLIN-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2815:
Assignee: Wang Cheng (was: hongbin ma)
> Empty results with prepareStatement but OK with KylinStatement
> --
>
> Key: KYLIN-2815
> URL: https://issues.apache.org/jira/browse/KYLIN-2815
> Project: Kylin
> Issue Type: Bug
> Components: Driver - JDBC
>Affects Versions: v2.1.0
>Reporter: youcheng.zhang
>Assignee: Wang Cheng
> Labels: scope
>
> Actually, my prepareStatement queries works well with kylin version v2.0.
> After I upgrade kylin to version v2.1, all queries have no results return,
> Then I write test：
> {code:java}
> Driver driver = (Driver)
> Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
> Properties info = new Properties();
> info.put("user", "ADMIN");
> info.put("password", "KYLIN");
> Connection conn = driver.connect("jdbc:kylin://IP:7070/project_name", info);
> PreparedStatement pst = conn.prepareStatement("select XX where XX = ? ")
> pst.setInt(1, 1);
> ResultSet resultSet = pst.executeQuery();
> {code}
> Return no results, and after that I change the query to statement
> {code:java}
> Statement state = conn.createStatement();
> ResultSet resultSet = state.executeQuery("select XX where XX = 1")
> {code}
> Finally, I got return values.
> I have no idea whether it is a bug or I have done something wrong with
> upgrade?
> As I have removed HBase tables “kylin_metadata_user” and
> “kylin_metadata_acl", I think
> I can't come back to version v2.0 any more:(
> I also try changed dependency kylin-jdbc to version 2.1.0 but help nothing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2806) Empty results from JDBC with Date filter in prepareStatement
[ https://issues.apache.org/jira/browse/KYLIN-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2806: Assignee: Wang Cheng (was: hongbin ma) > Empty results from JDBC with Date filter in prepareStatement > > > Key: KYLIN-2806 > URL: https://issues.apache.org/jira/browse/KYLIN-2806 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC, Query Engine >Affects Versions: v1.6.0, v2.0.0, v2.1.0 > Environment: kylin版本：apache-kylin-1.6.0-cdh5.7-bin > jdbc版本：kylin-jdbc-1.6.0.jar > java版本：java version "1.8.0_92" >Reporter: chengwansheng >Assignee: Wang Cheng > Labels: scope > > 用kylin的JDBC查询，查询条件是date类型的，如果通过prepareStatement、setDate是查询不出数据的，然而直接用SQL可以查出数据 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2862) BasicClientConnManager in RestClient can't do well with syncing many query severs
[
https://issues.apache.org/jira/browse/KYLIN-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159953#comment-16159953
]
Billy Liu commented on KYLIN-2862:
--
Hi [~whisper_deng], thanks for the patch. Could you make the parameter
configurable?
> BasicClientConnManager in RestClient can't do well with syncing many query
> severs
> -
>
> Key: KYLIN-2862
> URL: https://issues.apache.org/jira/browse/KYLIN-2862
> Project: Kylin
> Issue Type: Bug
> Components: General
>Affects Versions: v2.1.0
>Reporter: Fangyuan Deng
> Attachments: KYLIN-2862.0.patch
>
>
> In KYLIN-2814 , it fixs some kinds of sync wipe fails.
> After we use this patch , sync wipe fails still often happen, the log is
> like this.
> 2017-09-08 21:28:13,029 WARN [pool-10-thread-2] cachesync.Broadcaster:137 :
> Thread failed during wipe cache at BroadcastEvent{entity=cube, event=update,
> cacheKey=olap_nh_log_all_di_cube}, error msg:
> java.lang.IllegalStateException: Invalid use of BasicClientConnManager:
> connection still allocated.
> Make sure to release the connection before allocating another one.
> We search the code, then we found , the DefaultHttpClient uses
> BasicClientConnManager as default ConnManager.
> and the BasicClientConnManager is a simple connection manager that maintains
> only one connection at a time. Even though this class is thread-safe it ought
> to be used by one execution thread only.
> if we have many query servers, the BasicClientConnManager will be
> bottleneck.
> so we use PoolingClientConnectionManager instead, which can manages a pool
> of client connections and is able to service connection requests from
> multiple execution threads.
> Then everything goes fine. the code is below
> final PoolingClientConnectionManager cm = new
> PoolingClientConnectionManager();
> cm.setDefaultMaxPerRoute(20);
> cm.setMaxTotal(200);
> client = new DefaultHttpClient(cm, httpParams);
> see more about BasicClientConnManager and PoolingClientConnectionManager
> http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e619
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2863) fix the double caret bug in sample.sh when executed in old bash version of some enterprise OS
[
https://issues.apache.org/jira/browse/KYLIN-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu resolved KYLIN-2863.
--
Resolution: Fixed
Fix Version/s: (was: v2.1.0)
(was: v2.0.0)
v2.2.0
LGTM. Patch merged.
https://github.com/apache/kylin/commit/57f665d63aabc8b8cf886cf7562ca92effebd176
> fix the double caret bug in sample.sh when executed in old bash version of
> some enterprise OS
> -
>
> Key: KYLIN-2863
> URL: https://issues.apache.org/jira/browse/KYLIN-2863
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.0.0, v2.1.0
> Environment: OS: SuSE 11 sp4
> Hortonworks hdp 2.6.1.0-129
> hadoop 2.7.3
> hbase 1.1.2
> hive 1.2.1
> jdk 1.8.0
>Reporter: TING WANG
>Assignee: hongbin ma
> Labels: patch, starter
> Fix For: v2.2.0
>
> Attachments: fixbug-tina-double-caret.patch
>
>
> When an old version bash is used, the following error will occur:
> bq. /apache-kylin-2.0.0-bin/bin # ./sample.sh
> bq. Retrieving hadoop conf dir...
> bq. Retrieving hadoop conf dir...
> bq. KYLIN_HOME is set to /usr/share/kylin/apache-kylin-2.0.0-bin
> bq. Loading sample data into HDFS tmp path: /tmp/kylin/sample_cube/data
> bq. ./sample.sh: line 40: ${sample_database^^}: bad substitution
> bq. Going to create sample tables in hive to database default by cli
> bq. ..
> If you ignore the error above, another error will occur when building cube:
> bq. java.lang.IllegalStateException: No resource found at --
> /table/DEFAULT.KYLIN_SALES.json
> bq. at
> org.apache.kylin.engine.mr.common.AbstractHadoopJob.dumpResources(AbstractHadoopJob.java:550)
> bq. at
> org.apache.kylin.engine.mr.common.AbstractHadoopJob.dumpKylinPropsAndMetadata(AbstractHadoopJob.java:505)
> bq. at
> org.apache.kylin.engine.mr.common.AbstractHadoopJob.attachCubeMetadata(AbstractHadoopJob.java:458)
> bq. at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:122)
> bq. at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:102)
> bq. at
> org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:123)
> bq. at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
> bq. at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
> bq. at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
> bq. at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:142)
> bq. at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> bq. at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> bq. at java.lang.Thread.run(Thread.java:745)
> bq. result code:2
> The reason is that the Bash version in SuSE 11 sp4 enterprise is quite old:
> bq. #: bash -version
> bq. bash -version
> bq. GNU bash, version 3.2.57(1)-release (x86_64-suse-linux-gnu)
> bq. Copyright (C) 2007 Free Software Foundation, Inc.
> Instead of using double carets to convert the string to uppercase, we can use
> the tr command to do the same job as shown in this pr which has higher
> compatibility for various kinds of OS.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156508#comment-16156508
]
Billy Liu commented on KYLIN-2849:
--
Could you describe how the duplicate segments are generated? If there is a bug,
we should fix it first.
For the existing duplicated segments, you could export the metadata and remove
the wrong segments manually, and import the metadata back.
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
>Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
>Reporter: 翟玉勇
>Assignee: Dong Li
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/2017082200_2017082300;
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2841) LIMIT is buggy with subquery
[
https://issues.apache.org/jira/browse/KYLIN-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2841:
Assignee: zhengdong
> LIMIT is buggy with subquery
>
>
> Key: KYLIN-2841
> URL: https://issues.apache.org/jira/browse/KYLIN-2841
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>Assignee: zhengdong
> Attachments: 0001-KYLIN-2841-LIMIT-is-buggy-with-subquery.patch
>
>
> Hi, all.
> I found that limit in the web UI seems not behaving as expected.
> When I run a query like the follows:
> {code:sql}
> SELECT
> SUM(col3) AS col4,
> SUM(col5) AS total_col5,
> col1
> FROM
> (
> SELECT
> col1,
> col2,
> MAX(col3) AS col3,
> COUNT(*) AS col5
> FROM db.table
> WHERE col6 = 'somestring'
> GROUP BY col1, col2
> )
> GROUP BY col1
> {code}
> When I specify the limit as 50, the result has 19 records, and when I specify
> the limit as 50, there are 90+ records in the result and each record has
> higher col4 and total_col5.
> But for query that doesn't have subquery, the result remains the same no
> matter how I change the limit.
> I guess for the query with subquery, limit somehow limits the number of the
> result from the inner query instead of the result of the outer query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2841) LIMIT is buggy with subquery
[
https://issues.apache.org/jira/browse/KYLIN-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156505#comment-16156505
]
Billy Liu commented on KYLIN-2841:
--
Hi [~rogershi], could you help review the patch?
> LIMIT is buggy with subquery
>
>
> Key: KYLIN-2841
> URL: https://issues.apache.org/jira/browse/KYLIN-2841
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>Assignee: zhengdong
> Attachments: 0001-KYLIN-2841-LIMIT-is-buggy-with-subquery.patch
>
>
> Hi, all.
> I found that limit in the web UI seems not behaving as expected.
> When I run a query like the follows:
> {code:sql}
> SELECT
> SUM(col3) AS col4,
> SUM(col5) AS total_col5,
> col1
> FROM
> (
> SELECT
> col1,
> col2,
> MAX(col3) AS col3,
> COUNT(*) AS col5
> FROM db.table
> WHERE col6 = 'somestring'
> GROUP BY col1, col2
> )
> GROUP BY col1
> {code}
> When I specify the limit as 50, the result has 19 records, and when I specify
> the limit as 50, there are 90+ records in the result and each record has
> higher col4 and total_col5.
> But for query that doesn't have subquery, the result remains the same no
> matter how I change the limit.
> I guess for the query with subquery, limit somehow limits the number of the
> result from the inner query instead of the result of the outer query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156298#comment-16156298
]
Billy Liu commented on KYLIN-2849:
--
The Segment delete is only for the start and end segment, it's by design. For
more info, check https://issues.apache.org/jira/browse/KYLIN-1540
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
>Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
>Reporter: 翟玉勇
>Assignee: Dong Li
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/2017082200_2017082300;
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2315) Kylin run mode query not update metadata
[ https://issues.apache.org/jira/browse/KYLIN-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2315. -- Resolution: Not A Bug > Kylin run mode query not update metadata > > > Key: KYLIN-2315 > URL: https://issues.apache.org/jira/browse/KYLIN-2315 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: CentOS >Reporter: hoangle > > I'm running two kylin instance on two nodes in my cluster. > Two kylin point the same kylin.rest.servers=mycluster:7070 > Node1: I set kyin.server.mode=all ( for build cube and query) > Node2: I set kyin.server.mode=query ( for query only) > I face with problem when I finish build cube on Node 1, but then I move to > Node 2 I don't see any update about cube. > The cube data on Node 2 only update when I restart Kylin instance on Node 2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2315) Kylin run mode query not update metadata
[ https://issues.apache.org/jira/browse/KYLIN-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155023#comment-16155023 ] Billy Liu commented on KYLIN-2315: -- This is not an issue. This is the wrong configuration about kylin.rest.servers which has renamed into kylin.server.cluster-servers in Kylin 2.0 The "kylin.server.cluster-servers" should includes all nodes.For example kylin.server.cluster-servers=node1:7070,node2:7070 > Kylin run mode query not update metadata > > > Key: KYLIN-2315 > URL: https://issues.apache.org/jira/browse/KYLIN-2315 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 > Environment: CentOS >Reporter: hoangle > > I'm running two kylin instance on two nodes in my cluster. > Two kylin point the same kylin.rest.servers=mycluster:7070 > Node1: I set kyin.server.mode=all ( for build cube and query) > Node2: I set kyin.server.mode=query ( for query only) > I face with problem when I finish build cube on Node 1, but then I move to > Node 2 I don't see any update about cube. > The cube data on Node 2 only update when I restart Kylin instance on Node 2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2764) Build the dict for UHC column with MR
[ https://issues.apache.org/jira/browse/KYLIN-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151505#comment-16151505 ] Billy Liu commented on KYLIN-2764: -- +1. The improvement is quite impressive. > Build the dict for UHC column with MR > - > > Key: KYLIN-2764 > URL: https://issues.apache.org/jira/browse/KYLIN-2764 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: job-memory-after.png, job-memory-before.png > > > KYLIN-2217 has built dict for normal column with MR, but the UHC column > still build dict in JobServer. Like KYLIN-2217, we also could use MR build > dict for UHC column. which could thoroughly release the memory pressure and > improve job concurrent for JobServer as well as speed up multi UHC columns > procedure. > The MR input is the output of "Extract Fact Table Distinct Columns", the MR > output is the UHC column dict. Because it is very hard build global dict with > multi reducers, I use one reducer handle one UHC column and allocate enough > memory to the reducer. According to my test, 8G memory is enough. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2753. -- Resolution: Fixed Fix Version/s: v2.2.0 > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Fix For: v2.2.0 > > Attachments: > 0001-KYLIN-2753-Fix-for-the-special-case-that-jobs-not-re.patch, > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, > KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150328#comment-16150328 ] Billy Liu commented on KYLIN-2753: -- Thanks [~zhengd], patch merged. https://git1-us-west.apache.org/repos/asf?p=kylin.git;a=commit;h=1004a853 > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2753-Fix-for-the-special-case-that-jobs-not-re.patch, > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, > KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2824. -- Resolution: Fixed Fix Version/s: v2.2.0 Thanks Tianhao Zhao. The patch merged. https://git1-us-west.apache.org/repos/asf?p=kylin.git;a=commit;h=7a47e947 > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Fix For: v2.2.0 > > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150284#comment-16150284 ] Billy Liu commented on KYLIN-2753: -- Hi [~zhengd], could you rebase your patch with master branch first? > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, > KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2753: - Comment: was deleted (was: Thanks for the updates. But the patch merge check failed: git apply --check YLIN-2753-Job-duration-may-become-negative-after-th.patch warning: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java has type 100755, expected 100644 error: patch failed: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java:69 error: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java: patch does not apply warning: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java has type 100755, expected 100644 error: patch failed: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java:338 error: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java: patch does not apply) > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, > KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150281#comment-16150281 ] Billy Liu commented on KYLIN-2753: -- Thanks for the updates. But the patch merge check failed: git apply --check YLIN-2753-Job-duration-may-become-negative-after-th.patch warning: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java has type 100755, expected 100644 error: patch failed: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java:69 error: core-job/src/main/java/org/apache/kylin/job/execution/DefaultChainedExecutable.java: patch does not apply warning: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java has type 100755, expected 100644 error: patch failed: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java:338 error: core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java: patch does not apply > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, > KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2824) Parse Boolean type in JDBC driver
[ https://issues.apache.org/jira/browse/KYLIN-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148548#comment-16148548 ] Billy Liu commented on KYLIN-2824: -- Thanks the patch. LGTM. > Parse Boolean type in JDBC driver > - > > Key: KYLIN-2824 > URL: https://issues.apache.org/jira/browse/KYLIN-2824 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Reporter: Tianhao Zhao >Priority: Minor > Attachments: KYLIN-2824.patch > > > Exception below is thrown when querying kylin with Saiku as front end, or > when retrieving value from getBoolean method in jdbc driver. > That's because KylinClient in jdbc module dose not parse value properly when > dimension is boolean type. > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Boolean > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getBoolean(AbstractCursor.java:463) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$BooleanAccessor.getLong(AbstractCursor.java:468) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getInt(AbstractCursor.java:304) > at > org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaResultSet.getInt(AvaticaResultSet.java:252) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at > org.apache.commons.dbcp.DelegatingResultSet.getInt(DelegatingResultSet.java:225) > at mondrian.rolap.SqlStatement$4.get(SqlStatement.java:418) > at mondrian.rolap.SqlStatement$1.get(SqlStatement.java:394) > at > mondrian.rolap.SqlTupleReader$Target.internalAddRow(SqlTupleReader.java:260) > at > mondrian.rolap.SqlTupleReader$Target.addRow(SqlTupleReader.java:172) > at > mondrian.rolap.SqlTupleReader.prepareTuples(SqlTupleReader.java:545) > at mondrian.rolap.SqlTupleReader.readTuples(SqlTupleReader.java:690) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.executeList(RolapNativeSet.java:260) > at > mondrian.rolap.RolapNativeSet$SetEvaluator.execute(RolapNativeSet.java:200) > at > mondrian.olap.fun.NonEmptyCrossJoinFunDef$1.evaluateList(NonEmptyCrossJoinFunDef.java:80) > at > mondrian.calc.impl.AbstractListCalc.evaluateIterable(AbstractListCalc.java:71) > at > mondrian.olap.fun.CrossJoinFunDef$CrossJoinIterCalc.evaluateIterable(CrossJoinFunDef.java:184) > at mondrian.rolap.RolapResult.executeAxis(RolapResult.java:857) > at mondrian.rolap.RolapResult.evalLoad(RolapResult.java:693) > at mondrian.rolap.RolapResult.loadMembers(RolapResult.java:649) > at mondrian.rolap.RolapResult.(RolapResult.java:279) > at > mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:500) > ... 7 more -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2820) Query can't read window function's result from subquery
[
https://issues.apache.org/jira/browse/KYLIN-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146670#comment-16146670
]
Billy Liu commented on KYLIN-2820:
--
Thanks, [~visualskyrim]. This issue is well received.
> Query can't read window function's result from subquery
> ---
>
> Key: KYLIN-2820
> URL: https://issues.apache.org/jira/browse/KYLIN-2820
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>Assignee: Roger Shi
>
> I executed a query like the follows:
> {code:sql}
> select first_page_name, count(*) as page_name_count from
> (
> select first_value(page_name) over(partition by session_id) as
> first_page_name from some_db.some_table
> ) group by first_page_name;
> {code}
> This query resulted in one single record with an empty string as the
> first_page_name, and a big number as the page_name_count.
> However, when I ran the subquery without the outer query, it resulted in
> multiple records with different first_page_name, which proved the query above
> was wrong.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2820) Query can't read window function's result from subquery
[
https://issues.apache.org/jira/browse/KYLIN-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146506#comment-16146506
]
Billy Liu commented on KYLIN-2820:
--
Hi [~visualskyrim], could you reproduce this issue from sample cube?
> Query can't read window function's result from subquery
> ---
>
> Key: KYLIN-2820
> URL: https://issues.apache.org/jira/browse/KYLIN-2820
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.1.0
>Reporter: Mu Kong
>
> I executed a query like the follows:
> {code:sql}
> select first_page_name, count(*) as page_name_count from
> (
> select first_value(page_name) over(partition by session_id) as
> first_page_name from some_db.some_table
> ) group by first_page_name;
> {code}
> This query resulted in one single record with an empty string as the
> first_page_name, and a big number as the page_name_count.
> However, when I ran the subquery without the outer query, it resulted in
> multiple records with different first_page_name, which proved the query above
> was wrong.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-2816) Support hive unicode column name
Billy Liu created KYLIN-2816: Summary: Support hive unicode column name Key: KYLIN-2816 URL: https://issues.apache.org/jira/browse/KYLIN-2816 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v2.1.0 Reporter: Billy Liu Assignee: Billy Liu Currently, if the column name has non-english character, such as Chinese, the FlatTableJob will fail . The logs said character '汉字’ not supported here. Could use back-tick to wrap the column name as literature. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2815) No results with prepareStatement but OK with KylinStatement
[
https://issues.apache.org/jira/browse/KYLIN-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142520#comment-16142520
]
Billy Liu commented on KYLIN-2815:
--
Thanks for reporting. It's an issue, introduced by Prepare logic refactor in
the backend.
> No results with prepareStatement but OK with KylinStatement
> ---
>
> Key: KYLIN-2815
> URL: https://issues.apache.org/jira/browse/KYLIN-2815
> Project: Kylin
> Issue Type: Bug
> Components: Driver - JDBC
>Affects Versions: v2.1.0
>Reporter: youcheng.zhang
>Assignee: hongbin ma
>
> Actually, my prepareStatement queries works well with kylin version v2.0.
> After I upgrade kylin to version v2.1, all queries have no results return,
> Then I write test：
> {code:java}
> Driver driver = (Driver)
> Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
> Properties info = new Properties();
> info.put("user", "ADMIN");
> info.put("password", "KYLIN");
> Connection conn = driver.connect("jdbc:kylin://IP:7070/project_name", info);
> PreparedStatement pst = conn.prepareStatement("select XX where XX = ? ")
> pst.setInt(1, 1);
> ResultSet resultSet = pst.executeQuery();
> {code}
> Return no results, and after that I change the query to statement
> {code:java}
> Statement state = conn.createStatement();
> ResultSet resultSet = state.executeQuery("select XX where XX = 1")
> {code}
> Finally, I got return values.
> I have no idea whether it is a bug or I have done something wrong with
> upgrade?
> As I have removed HBase tables “kylin_metadata_user” and
> “kylin_metadata_acl", I think
> I can't come back to version v2.0 any more:(
> I also try changed dependency kylin-jdbc to version 2.1.0 but help nothing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2815) No results with prepareStatement but OK with KylinStatement
[
https://issues.apache.org/jira/browse/KYLIN-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2815:
Assignee: hongbin ma
> No results with prepareStatement but OK with KylinStatement
> ---
>
> Key: KYLIN-2815
> URL: https://issues.apache.org/jira/browse/KYLIN-2815
> Project: Kylin
> Issue Type: Bug
> Components: Driver - JDBC
>Affects Versions: v2.1.0
>Reporter: youcheng.zhang
>Assignee: hongbin ma
>
> Actually, my prepareStatement queries works well with kylin version v2.0.
> After I upgrade kylin to version v2.1, all queries have no results return,
> Then I write test：
> {code:java}
> Driver driver = (Driver)
> Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
> Properties info = new Properties();
> info.put("user", "ADMIN");
> info.put("password", "KYLIN");
> Connection conn = driver.connect("jdbc:kylin://IP:7070/project_name", info);
> PreparedStatement pst = conn.prepareStatement("select XX where XX = ? ")
> pst.setInt(1, 1);
> ResultSet resultSet = pst.executeQuery();
> {code}
> Return no results, and after that I change the query to statement
> {code:java}
> Statement state = conn.createStatement();
> ResultSet resultSet = state.executeQuery("select XX where XX = 1")
> {code}
> Finally, I got return values.
> I have no idea whether it is a bug or I have done something wrong with
> upgrade?
> As I have removed HBase tables “kylin_metadata_user” and
> “kylin_metadata_acl", I think
> I can't come back to version v2.0 any more:(
> I also try changed dependency kylin-jdbc to version 2.1.0 but help nothing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-2812) Save to wrong database when loading Kafka Topic
Billy Liu created KYLIN-2812: Summary: Save to wrong database when loading Kafka Topic Key: KYLIN-2812 URL: https://issues.apache.org/jira/browse/KYLIN-2812 Project: Kylin Issue Type: Bug Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Fix For: v2.2.0 When loading Kafka Topic, user could select the destination database name. Currently, all topics are saved into DEFAULT database, which is not expected. Here is a bug during reading create table parameter. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2286) global snapshot table for one cube
[ https://issues.apache.org/jira/browse/KYLIN-2286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139845#comment-16139845 ] Billy Liu commented on KYLIN-2286: -- I think this is Slowly Changing Dimension(SCD) topic. For more info, check http://datawarehouse4u.info/SCD-Slowly-Changing-Dimensions.html The issue becomes more complicated when dealing with normal dimension and derived dimension in lookup table. To support Type 0 - The passive method, use normal dimension in lookup table. To support Type 1 - Overwriting the old value, use derived dimension in lookup table. To support Type 0 with derived dimension will use global snapshot table like this issue. This feature is welcomed, to support more flexible Kylin. > global snapshot table for one cube > --- > > Key: KYLIN-2286 > URL: https://issues.apache.org/jira/browse/KYLIN-2286 > Project: Kylin > Issue Type: Improvement >Reporter: fengYu >Assignee: fengYu > > I current version, Kylin build a snapshot table for a segment and isolate > with each other in the same cube, even though some segments share the same > snapshot table storage . > I some scene， we need global snapshot table for one cube, such as we has a > cube with snapshot table,ID is PK，the first day, the table look like: > id name > 1 A > 2 B > 3 C > the query 'select name, count(1) from fact join dimension group by name' get > result: > A xx > B xx > C xx > the next day(segment), lookup table modified, it looks like : > id name > 1 A > 2 D > 3 E > the same query return : > A xx > B xx > C xx > D xx > E xx > However B and D, C and E has the same ID, we need the newest result. so a > global snapshot table shared by all segments which has always the newest > values is needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2810) Kylin UDF support
[ https://issues.apache.org/jira/browse/KYLIN-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139740#comment-16139740 ] Billy Liu commented on KYLIN-2810: -- Thanks [~feng_xiao_yu], the UDF extensions are highly welcomed. > Kylin UDF support > - > > Key: KYLIN-2810 > URL: https://issues.apache.org/jira/browse/KYLIN-2810 > Project: Kylin > Issue Type: New Feature >Reporter: fengYu >Assignee: fengYu > > Kylin do not support some function calcite do not support, May I contribute > some UDF in kylin, In this way, some of our BI tools can use kylin everywhere. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2810) Kylin UDF support
[ https://issues.apache.org/jira/browse/KYLIN-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2810: Assignee: fengYu Issue Type: New Feature (was: Bug) > Kylin UDF support > - > > Key: KYLIN-2810 > URL: https://issues.apache.org/jira/browse/KYLIN-2810 > Project: Kylin > Issue Type: New Feature >Reporter: fengYu >Assignee: fengYu > > Kylin do not support some function calcite do not support, May I contribute > some UDF in kylin, In this way, some of our BI tools can use kylin everywhere. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2806) No results from JDBC with Date filter in prepareStatement
[ https://issues.apache.org/jira/browse/KYLIN-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2806: Assignee: hongbin ma (was: liyang) > No results from JDBC with Date filter in prepareStatement > - > > Key: KYLIN-2806 > URL: https://issues.apache.org/jira/browse/KYLIN-2806 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC, Query Engine >Affects Versions: v1.6.0, v2.0.0, v2.1.0 > Environment: kylin版本：apache-kylin-1.6.0-cdh5.7-bin > jdbc版本：kylin-jdbc-1.6.0.jar > java版本：java version "1.8.0_92" >Reporter: chengwansheng >Assignee: hongbin ma > > 用kylin的JDBC查询，查询条件是date类型的，如果通过prepareStatement、setDate是查询不出数据的，然而直接用SQL可以查出数据 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2804) Unchecked return value from Input#read() in PercentileCounterSerializer
[
https://issues.apache.org/jira/browse/KYLIN-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2804:
Assignee: zhengdong
> Unchecked return value from Input#read() in PercentileCounterSerializer
> ---
>
> Key: KYLIN-2804
> URL: https://issues.apache.org/jira/browse/KYLIN-2804
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: zhengdong
>Priority: Minor
>
> {code}
> byte[] buffer = new byte[length];
> input.read(buffer);
> {code}
> length bytes are allocated.
> The return value from read() should be checked against length.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2799) Building cube with percentile measure encounter with NullPointerException
[
https://issues.apache.org/jira/browse/KYLIN-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu resolved KYLIN-2799.
--
Resolution: Fixed
Fix Version/s: v2.2.0
Thanks [~zhengd], LGTM. The patch is merged.
https://git1-us-west.apache.org/repos/asf?p=kylin.git;a=commit;h=aaf94169
> Building cube with percentile measure encounter with NullPointerException
> -
>
> Key: KYLIN-2799
> URL: https://issues.apache.org/jira/browse/KYLIN-2799
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0.0, v2.1.0
>Reporter: zhengdong
>Assignee: zhengdong
> Fix For: v2.2.0
>
> Attachments:
> 0001-KYLIN-2799-Building-cube-with-percentile-measure-enc.patch
>
>
> The error trace log as follows:
> {code:java}
> at
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at scala.Option.foreach(Option.scala:257)
> at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1925)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1938)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1158)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1005)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:996)
> at
> org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:823)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.saveToHDFS(SparkCubingByLayer.java:298)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:257)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.job.shaded.com.tdunning.math.stats.AbstractTDigest.add(AbstractTDigest.java:160)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.merge(PercentileCounter.java:63)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.(PercentileCounter.java:44)
> at
> org.apache.kylin.measure.percentile.PercentileAggregator.aggregate(PercentileAggregator.java:46)
> at
>
[jira] [Commented] (KYLIN-2799) Building cube with percentile measure encounter with NullPointerException
[
https://issues.apache.org/jira/browse/KYLIN-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136160#comment-16136160
]
Billy Liu commented on KYLIN-2799:
--
Hi [~zhengd], why com.tdunning.math.stats library was removed from Kyro.
> Building cube with percentile measure encounter with NullPointerException
> -
>
> Key: KYLIN-2799
> URL: https://issues.apache.org/jira/browse/KYLIN-2799
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0.0, v2.1.0
>Reporter: zhengdong
>Assignee: zhengdong
> Attachments:
> 0001-KYLIN-2799-Building-cube-with-percentile-measure-enc.patch
>
>
> The error trace log as follows:
> {code:java}
> at
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at scala.Option.foreach(Option.scala:257)
> at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1925)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1938)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1158)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1005)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:996)
> at
> org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:823)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.saveToHDFS(SparkCubingByLayer.java:298)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:257)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.job.shaded.com.tdunning.math.stats.AbstractTDigest.add(AbstractTDigest.java:160)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.merge(PercentileCounter.java:63)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.(PercentileCounter.java:44)
> at
> org.apache.kylin.measure.percentile.PercentileAggregator.aggregate(PercentileAggregator.java:46)
> at
>
[jira] [Assigned] (KYLIN-2800) All dictionaries should be built based on the flat hive table
[ https://issues.apache.org/jira/browse/KYLIN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2800: Assignee: zhengdong > All dictionaries should be built based on the flat hive table > - > > Key: KYLIN-2800 > URL: https://issues.apache.org/jira/browse/KYLIN-2800 > Project: Kylin > Issue Type: Bug >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2800-All-dictionaries-should-be-built-based-on.patch > > > After KYLIN-2457, we still got wrong query result sometimes after a merging > job finished. > Finally, we realize the root cause is that we always use lookup table as > source data to build dictionaries for FK columns. > However, incremental lookup table doesn't mean sequential and incremental PK. > If a new record inserted into the lookup table while its PK column does not > have the max value, ID numbers in the new dictionary could be changed for > those PK value larger than the newest one. What's more, using lookup table as > source data for FK column's dictionary may has performance advantage for > merging job, but also may encounter too big dictionary problem for large > lookup tables. And we must add some validation rules to ensure the PK value > sequential and incremental. > On the another hand, we could just unify using the flat hive table as data > source for all dictionaries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2800) All dictionaries should be built based on the flat hive table
[ https://issues.apache.org/jira/browse/KYLIN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136081#comment-16136081 ] Billy Liu commented on KYLIN-2800: -- Good found. > All dictionaries should be built based on the flat hive table > - > > Key: KYLIN-2800 > URL: https://issues.apache.org/jira/browse/KYLIN-2800 > Project: Kylin > Issue Type: Bug >Reporter: zhengdong > Attachments: > 0001-KYLIN-2800-All-dictionaries-should-be-built-based-on.patch > > > After KYLIN-2457, we still got wrong query result sometimes after a merging > job finished. > Finally, we realize the root cause is that we always use lookup table as > source data to build dictionaries for FK columns. > However, incremental lookup table doesn't mean sequential and incremental PK. > If a new record inserted into the lookup table while its PK column does not > have the max value, ID numbers in the new dictionary could be changed for > those PK value larger than the newest one. What's more, using lookup table as > source data for FK column's dictionary may has performance advantage for > merging job, but also may encounter too big dictionary problem for large > lookup tables. And we must add some validation rules to ensure the PK value > sequential and incremental. > On the another hand, we could just unify using the flat hive table as data > source for all dictionaries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2799) Building cube with percentile measure encounter with NullPointerException
[
https://issues.apache.org/jira/browse/KYLIN-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2799:
Assignee: zhengdong
> Building cube with percentile measure encounter with NullPointerException
> -
>
> Key: KYLIN-2799
> URL: https://issues.apache.org/jira/browse/KYLIN-2799
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0.0, v2.1.0
>Reporter: zhengdong
>Assignee: zhengdong
> Attachments:
> 0001-KYLIN-2799-Building-cube-with-percentile-measure-enc.patch
>
>
> The error trace log as follows:
> {code:java}
> at
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)
> at scala.Option.foreach(Option.scala:257)
> at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> at
> org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1925)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1938)
> at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1158)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1005)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:996)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:996)
> at
> org.apache.spark.api.java.JavaPairRDD.saveAsNewAPIHadoopFile(JavaPairRDD.scala:823)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.saveToHDFS(SparkCubingByLayer.java:298)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:257)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.job.shaded.com.tdunning.math.stats.AbstractTDigest.add(AbstractTDigest.java:160)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.merge(PercentileCounter.java:63)
> at
> org.apache.kylin.measure.percentile.PercentileCounter.(PercentileCounter.java:44)
> at
> org.apache.kylin.measure.percentile.PercentileAggregator.aggregate(PercentileAggregator.java:46)
> at
> org.apache.kylin.measure.percentile.PercentileAggregator.aggregate(PercentileAggregator.java:23)
> at
>
[jira] [Assigned] (KYLIN-2779) Segment size estimation is wrong for bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2779: Assignee: Alexander Sterligov > Segment size estimation is wrong for bitmap measure > --- > > Key: KYLIN-2779 > URL: https://issues.apache.org/jira/browse/KYLIN-2779 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.0.0 >Reporter: Alexander Sterligov >Assignee: Alexander Sterligov > > If bitmap is used for distinct count, then size estimation is wrong for about > 4 times. > The problem is fixed by kylin.cube.size-estimate-countdistinct-ratio=0.2. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2707) Fix NPE in JobInfoConverter
[ https://issues.apache.org/jira/browse/KYLIN-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2707: - Summary: Fix NPE in JobInfoConverter (was: Fxi NPE in JobInfoConverter) > Fix NPE in JobInfoConverter > --- > > Key: KYLIN-2707 > URL: https://issues.apache.org/jira/browse/KYLIN-2707 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Fix For: v2.1.0 > > Attachments: KYLIN-2707.patch > > > The other day, I couldn't get all job info because the stepOutput for one > job in JobInfoConverter.parseToJobStep is NULL, I didn't dive into why > stepOutput is NULL, but since stepOutput could be NULL, I think we should > handle it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2757) Get cube size when using Azure Data Lake Store
[ https://issues.apache.org/jira/browse/KYLIN-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2757: - Issue Type: Improvement (was: Bug) Summary: Get cube size when using Azure Data Lake Store (was: Couldn't get cube size if using Azure Data Lake Store) > Get cube size when using Azure Data Lake Store > -- > > Key: KYLIN-2757 > URL: https://issues.apache.org/jira/browse/KYLIN-2757 > Project: Kylin > Issue Type: Improvement >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI > Fix For: v2.1.0 > > > Similar as KYLIN-2490, Azure Data Lake Store uses different counters for > jobs. for examples, ADL: Number of bytes written (key: "ADL_BYTES_WRITTEN") -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2754) Fix Sync issue when reload existing hive table
[ https://issues.apache.org/jira/browse/KYLIN-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2754: - Summary: Fix Sync issue when reload existing hive table (was: Sync issue when reload existing hive table) > Fix Sync issue when reload existing hive table > -- > > Key: KYLIN-2754 > URL: https://issues.apache.org/jira/browse/KYLIN-2754 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2754-Sync-issue-when-reload-existing-hive-tabl.patch > > > When loading hive table, after putting the TableExtDesc to resource store, we > should also update local and remote TableExtDesc cache to avoid overwriting > conflict exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2753) Job duration may become negative after the job resumed from error or paused status
[ https://issues.apache.org/jira/browse/KYLIN-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2753: - Fix Version/s: (was: v2.1.0) > Job duration may become negative after the job resumed from error or paused > status > -- > > Key: KYLIN-2753 > URL: https://issues.apache.org/jira/browse/KYLIN-2753 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2753-Job-duration-may-become-negative-after-th.patch, Screen Shot > 2017-08-02 at 13.30.51.png, Screen Shot 2017-08-02 at 14.01.13.png > > > After KYLIN-2655, we calculate the 'Interrupt time' when a job resumed from > error/stopped status to give more accurate job duration time. However, when > calculate the real duration time = 'Total time' - 'Interrupt time', we could > get a negative value. > The root cause is 'Interrupt time' calculated in > DefaultChainedExecutable.onExecuteStart method. The 'Interrupt time' value is > not only added with the true 'Interrupt time' when the job scheduled after > resumed, but also increased when subsequent sub tasks executed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2742) Specify login page for Spring security 4.x
[ https://issues.apache.org/jira/browse/KYLIN-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2742: - Issue Type: Improvement (was: Bug) Summary: Specify login page for Spring security 4.x (was: Spring security 4.x will use a dynamically generated default login page when no login page is specified.) > Specify login page for Spring security 4.x > -- > > Key: KYLIN-2742 > URL: https://issues.apache.org/jira/browse/KYLIN-2742 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2742-Spring-security-4.x-will-use-a-dynamicall.patch, > 2017-07-14.PNG > > > Spring security 4.x will use a dynamically generated default login page when > no login page is specified. > The reproducible steps are as follows: > Use link 'http://xxx:7070/kylin/login' to access kylin, or refresh on login > page, will jump to spring security dynamically generated login page. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2713) Fix ITJdbcSourceTableLoaderTest.java and ITJdbcTableReaderTest.java missing license header
[ https://issues.apache.org/jira/browse/KYLIN-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2713: - Summary: Fix ITJdbcSourceTableLoaderTest.java and ITJdbcTableReaderTest.java missing license header (was: Kylin failed to compile，ITJdbcSourceTableLoaderTest.java and ITJdbcTableReaderTest.java missing license) > Fix ITJdbcSourceTableLoaderTest.java and ITJdbcTableReaderTest.java missing > license header > -- > > Key: KYLIN-2713 > URL: https://issues.apache.org/jira/browse/KYLIN-2713 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Fix For: v2.1.0 > > Attachments: 0001-KYLIN-2713-Kylin-failed-to-compile.patch > > > Kylin failed to compile，ITJdbcSourceTableLoaderTest.java and > ITJdbcTableReaderTest.java missing license > [ERROR] > \jenkins\workspace\Kylin\kylin-it\src\test\java\org\apache\kylin\source\jdbc\ITJdbcSourceTableLoaderTest.java:1: > 当前行与被期待的 header: '/*'不符?? [Header] > [ERROR] > \jenkins\workspace\Kylin\kylin-it\src\test\java\org\apache\kylin\source\jdbc\ITJdbcTableReaderTest.java:1: > 当前行与被期待的 header: '/*' 不符??[Header] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2696) Check SQL injection in model filter condition
[ https://issues.apache.org/jira/browse/KYLIN-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2696: - Issue Type: Improvement (was: Bug) > Check SQL injection in model filter condition > - > > Key: KYLIN-2696 > URL: https://issues.apache.org/jira/browse/KYLIN-2696 > Project: Kylin > Issue Type: Improvement >Reporter: Xiaqing Wang >Assignee: Xiaqing Wang > Fix For: v2.1.0 > > > We should check the model filter condition in case of someone make use of it > to do SQL injection to Hive. > Since it is a String embed into a WHERE clause, we simply forbid it to > include ';' character, except it is within a pair of quotations. > Resolved. Please refer to: > commit b7f72c5f407ab3eaf4e133dc08172d56fece49d1 > Author: Xiaqing <454530...@qq.com> > Date: Fri Jun 30 09:42:07 2017 +0800 > KYLIN-2696 Check SQL injection in filter condition -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2691) Support delete broken cube
[ https://issues.apache.org/jira/browse/KYLIN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2691: - Issue Type: Improvement (was: Bug) Summary: Support delete broken cube (was: Broken cube cannot be deleted) > Support delete broken cube > -- > > Key: KYLIN-2691 > URL: https://issues.apache.org/jira/browse/KYLIN-2691 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: jiatao.tao >Assignee: luguosheng >Priority: Minor > Fix For: v2.1.0 > > > Broken cube cannot be deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2684) Fix Object.class not registered in Kyro issue with spark cubing
[ https://issues.apache.org/jira/browse/KYLIN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2684: - Summary: Fix Object.class not registered in Kyro issue with spark cubing (was: Fix Object.class not registeredFailed to build cube with spark engine) > Fix Object.class not registered in Kyro issue with spark cubing > --- > > Key: KYLIN-2684 > URL: https://issues.apache.org/jira/browse/KYLIN-2684 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.1.0 > Environment: spark 1.6.3 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Blocker > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2684-Failed-to-build-cube-with-spark-engine.patch > > > Failed to build cube with spark engine at "#7 Step Name: Build Cube with > Spark". > The error log is as following: > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: com.esotericsoftware.kryo.KryoException: > java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > Serialization trace: > cuboidTreeLock (org.apache.kylin.cube.model.CubeDesc) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568) > at > org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194) > at > org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:203) > at > org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:102) > at > org.apache.spark.broadcast.TorrentBroadcast.(TorrentBroadcast.scala:85) > at > org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34) > at > org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63) > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326) > at > org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:639) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:166) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:442) > at > com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:79) > at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:472) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:561) > ... 22 more > > > > Change 'private Object cuboidTreeLock = new Object();' to 'private byte[] > cuboidTreeLock = new byte[0];' in CubeDesc.java to solve this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2684) Fix Object.class not registeredFailed to build cube with spark engine
[ https://issues.apache.org/jira/browse/KYLIN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2684: Assignee: peng.jianhua (was: Billy Liu) > Fix Object.class not registeredFailed to build cube with spark engine > - > > Key: KYLIN-2684 > URL: https://issues.apache.org/jira/browse/KYLIN-2684 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.1.0 > Environment: spark 1.6.3 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Blocker > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2684-Failed-to-build-cube-with-spark-engine.patch > > > Failed to build cube with spark engine at "#7 Step Name: Build Cube with > Spark". > The error log is as following: > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: com.esotericsoftware.kryo.KryoException: > java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > Serialization trace: > cuboidTreeLock (org.apache.kylin.cube.model.CubeDesc) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568) > at > org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194) > at > org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:203) > at > org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:102) > at > org.apache.spark.broadcast.TorrentBroadcast.(TorrentBroadcast.scala:85) > at > org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34) > at > org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63) > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326) > at > org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:639) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:166) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:442) > at > com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:79) > at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:472) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:561) > ... 22 more > > > > Change 'private Object cuboidTreeLock = new Object();' to 'private byte[] > cuboidTreeLock = new byte[0];' in CubeDesc.java to solve this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2684) Fix Object.class not registeredFailed to build cube with spark engine
[ https://issues.apache.org/jira/browse/KYLIN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2684: Assignee: Billy Liu (was: peng.jianhua) > Fix Object.class not registeredFailed to build cube with spark engine > - > > Key: KYLIN-2684 > URL: https://issues.apache.org/jira/browse/KYLIN-2684 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.1.0 > Environment: spark 1.6.3 >Reporter: peng.jianhua >Assignee: Billy Liu >Priority: Blocker > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2684-Failed-to-build-cube-with-spark-engine.patch > > > Failed to build cube with spark engine at "#7 Step Name: Build Cube with > Spark". > The error log is as following: > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: com.esotericsoftware.kryo.KryoException: > java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > Serialization trace: > cuboidTreeLock (org.apache.kylin.cube.model.CubeDesc) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568) > at > org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194) > at > org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:203) > at > org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:102) > at > org.apache.spark.broadcast.TorrentBroadcast.(TorrentBroadcast.scala:85) > at > org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34) > at > org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63) > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326) > at > org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:639) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:166) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:442) > at > com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:79) > at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:472) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:561) > ... 22 more > > > > Change 'private Object cuboidTreeLock = new Object();' to 'private byte[] > cuboidTreeLock = new byte[0];' in CubeDesc.java to solve this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2684) Fix Object.class not registeredFailed to build cube with spark engine
[ https://issues.apache.org/jira/browse/KYLIN-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2684: - Summary: Fix Object.class not registeredFailed to build cube with spark engine (was: Failed to build cube with spark engine) > Fix Object.class not registeredFailed to build cube with spark engine > - > > Key: KYLIN-2684 > URL: https://issues.apache.org/jira/browse/KYLIN-2684 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v2.1.0 > Environment: spark 1.6.3 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Blocker > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2684-Failed-to-build-cube-with-spark-engine.patch > > > Failed to build cube with spark engine at "#7 Step Name: Build Cube with > Spark". > The error log is as following: > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: com.esotericsoftware.kryo.KryoException: > java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > Serialization trace: > cuboidTreeLock (org.apache.kylin.cube.model.CubeDesc) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:585) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213) > at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568) > at > org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194) > at > org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:203) > at > org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:102) > at > org.apache.spark.broadcast.TorrentBroadcast.(TorrentBroadcast.scala:85) > at > org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34) > at > org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:63) > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1326) > at > org.apache.spark.api.java.JavaSparkContext.broadcast(JavaSparkContext.scala:639) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:166) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Class is not registered: Object > Note: To register this class use: kryo.register(Object.class); > at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:442) > at > com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:79) > at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:472) > at > com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:561) > ... 22 more > > > > Change 'private Object cuboidTreeLock = new Object();' to 'private byte[] > cuboidTreeLock = new byte[0];' in CubeDesc.java to solve this problem. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2678) Fix minor issues in KylinConfigCLITest
[ https://issues.apache.org/jira/browse/KYLIN-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2678: - Summary: Fix minor issues in KylinConfigCLITest (was: found error in test case KylinConfigCLITest) > Fix minor issues in KylinConfigCLITest > -- > > Key: KYLIN-2678 > URL: https://issues.apache.org/jira/browse/KYLIN-2678 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v2.0.0 > Environment: win7-x64 >Reporter: wangxianbin >Assignee: wangxianbin >Priority: Minor > Labels: newbie > Fix For: v2.1.0 > > Attachments: > master-0001-KYLIN-2678-fix-error-in-test-case-KylinConfigCLITest.patch, > master-hbase0.98-0001-KYLIN-2678-fix-error-in-test-case-KylinConfigCLITest.patch > > > two issues found in test case KylinConfigCLITest: > # PrintStream should be closed explicitly for avoiding local temp file > cleanup fail > # System.lineSeparator() should be used instead of hard code by '\n' for OS > compatibility -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2677) Add project configuration view page
[ https://issues.apache.org/jira/browse/KYLIN-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2677: - Issue Type: Improvement (was: Bug) Summary: Add project configuration view page (was: There is no place to view the project configuration and can only be viewed on the edit project page.) > Add project configuration view page > --- > > Key: KYLIN-2677 > URL: https://issues.apache.org/jira/browse/KYLIN-2677 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2677-There-is-no-place-to-view-the-project-con.patch, before.PNG, > modify.PNG > > > There is no place to view the project configuration and can only be viewed on > the edit project page. > Add a tab to make it easier for users to view the project configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2673) Support change the fact table when the cube is disable
[ https://issues.apache.org/jira/browse/KYLIN-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2673: - Issue Type: Improvement (was: Bug) Summary: Support change the fact table when the cube is disable (was: Should allow user to change fact table as long as the cube is disable) > Support change the fact table when the cube is disable > -- > > Key: KYLIN-2673 > URL: https://issues.apache.org/jira/browse/KYLIN-2673 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v2.1.0 > > > Currently, user couldn't change fact table even though the cube is disable, > which isn't reasonable. We should allow user to change fact table as long as > the cube is disable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2670) Fix CASE WHEN issue in orderby clause
[ https://issues.apache.org/jira/browse/KYLIN-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2670: - Summary: Fix CASE WHEN issue in orderby clause (was: CASE WHEN supporting problem in kylin2.0) > Fix CASE WHEN issue in orderby clause > - > > Key: KYLIN-2670 > URL: https://issues.apache.org/jira/browse/KYLIN-2670 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.0.0 >Reporter: zhou degao >Assignee: liyang > Labels: focus, scope > Fix For: v2.1.0 > > > Following query failed in kylin 2.0 but succeeded in kylin 1.6 > select "fact_pv_data_alias"."PRODUCT_NAME" as "c0", > "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC > Reported error in kylin 2.0: > Error while executing SQL "select "fact_pv_data_alias"."PRODUCT_NAME" as > "c0", "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC LIMIT 5": index (2) must be less than > size (2) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2668) Support Calcites Properties in JDBC URL
[ https://issues.apache.org/jira/browse/KYLIN-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2668: - Issue Type: Improvement (was: Bug) Summary: Support Calcites Properties in JDBC URL (was: Kylin JDBC Driver doesn't support Calcites Properties in URL) > Support Calcites Properties in JDBC URL > --- > > Key: KYLIN-2668 > URL: https://issues.apache.org/jira/browse/KYLIN-2668 > Project: Kylin > Issue Type: Improvement > Components: Driver - JDBC >Affects Versions: 1.x-HBase1.1.3 >Reporter: Joe Swingle >Assignee: Joe Swingle > Fix For: v2.1.0 > > Attachments: KYLIN-2668-v2.patch > > > We currently access our OLAP Cubes in Kylin from our BI Tool over JDBC. We > run Kylin over SSL. When using our COTS BI Tool, we cannot programmatically > pass in property values such as ssl=True into a java.util.Properties. We > can only specify a Username, Password and URL. > According to Calcite we should be able to provide some properties in the JDBC > URL. Line 142 of org.apache.calcite.avatica.UnregisteredDriver, builds java > properties based on the URL. This works for Kylin. For example, > "jdbc:kylin:ssl=True;//kylinserver.com/Projectname" Will correctly > establish the connection as using SSL. > The problem is in KylinConnection's Constructor. It splits the jdbcURL on > "jdbc:kylin://". So, it ignores the SSL. (KylinConnection. Line 54-60.) > The KylinConnection needs to be updated to support this method for setting > properties, so it can be used by 3rd party applications where programing the > connection is not possible. > We have coded this locally, and will submit patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2664) Fix Extended column bug in web
[
https://issues.apache.org/jira/browse/KYLIN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2664:
-
Description: The option for {{Extended column on fact table}} should be
{{getCommonMetricColumns()}} not {{getAllModelDimColumns()}} (was: The option
for {{Extended column on fact table}} should be {{getCommonMetricColumns()}}
not {{getCommonMetricColumns()}})
> Fix Extended column bug in web
> --
>
> Key: KYLIN-2664
> URL: https://issues.apache.org/jira/browse/KYLIN-2664
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
> Fix For: v2.1.0
>
>
> The option for {{Extended column on fact table}} should be
> {{getCommonMetricColumns()}} not {{getAllModelDimColumns()}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2661) Fix Cube list page display issue when using MODELER or ANALYST
[ https://issues.apache.org/jira/browse/KYLIN-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2661: - Summary: Fix Cube list page display issue when using MODELER or ANALYST (was: Cube list page has some minor problems when using MODELER or ANALYST users.) > Fix Cube list page display issue when using MODELER or ANALYST > -- > > Key: KYLIN-2661 > URL: https://issues.apache.org/jira/browse/KYLIN-2661 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2661-Cube-list-page-has-some-minor-problems-wh.patch, > 2017-06-08.PNG > > > 1.The Action button and "N/A" are both shown. > 2.Show more than one Action button -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2660) Show error tips if load hive error occurs and can not be connected.
[ https://issues.apache.org/jira/browse/KYLIN-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2660: - Summary: Show error tips if load hive error occurs and can not be connected. (was: "Load Hive Table From Tree" page always show "Loading Databases" without error tips, when hive error occurs and can not be connected.) > Show error tips if load hive error occurs and can not be connected. > --- > > Key: KYLIN-2660 > URL: https://issues.apache.org/jira/browse/KYLIN-2660 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2660-Load-Hive-Table-From-Tree-page-always-sho.patch, > 2016-06-07.PNG > > > "Load Hive Table From Tree" page always show "Loading Databases" without > error tips, when hive error occurs and can not be connected. > 2017-06-07 14:59:08,419 ERROR [http-bio-7070-exec-5] > controller.TableController:181 : Could not connect to meta store using any of > the URIs provided. Most recent failure: > org.apache.thrift.transport.TTransportException: java.net.ConnectException: > Connection refused -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2657) Fix Cube merge NPE whose TopN dictionary not found
[ https://issues.apache.org/jira/browse/KYLIN-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2657: - Summary: Fix Cube merge NPE whose TopN dictionary not found (was: Cube merge TopN dictionary not found, cause NPE) > Fix Cube merge NPE whose TopN dictionary not found > -- > > Key: KYLIN-2657 > URL: https://issues.apache.org/jira/browse/KYLIN-2657 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Roger Shi >Assignee: Roger Shi > Fix For: v2.1.0 > > > One TopN measure with two group-by columns, and both of them are in fact > table. If one cardinality is high and the other is low, merging Cube may fail > for NPE (dict not found). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2655) Fix wrong job duration issue when resuming the error or stopped job.
[ https://issues.apache.org/jira/browse/KYLIN-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2655: - Summary: Fix wrong job duration issue when resuming the error or stopped job. (was: There are some minor problems with the duration of the job when resuming the error job or stopped job.) > Fix wrong job duration issue when resuming the error or stopped job. > > > Key: KYLIN-2655 > URL: https://issues.apache.org/jira/browse/KYLIN-2655 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.0.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2655-There-are-some-minor-problems-with-the-du.patch > > > 1.After the error job is resumed, the duration of the job is no longer > changed until the job is finished. > 2.Duration contains the time from job error to job resume, not the actual > execution time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2644) Fix "Add Project" after refreshing Insight page
[ https://issues.apache.org/jira/browse/KYLIN-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2644: - Summary: Fix "Add Project" after refreshing Insight page (was: When I refresh the insight page and click "Add Project" button, but the new project pop-up page is not showing.) > Fix "Add Project" after refreshing Insight page > --- > > Key: KYLIN-2644 > URL: https://issues.apache.org/jira/browse/KYLIN-2644 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2644-When-I-refresh-the-insight-page-and-click.patch, > 2017-05-25.PNG > > > After I refresh the insight page, I click "Add Project" button, the new > project pop-up page is not showing. > web browser : chrome 49.0.2623.75, firefox 46.0 > The reproducible steps are as follows: > 1.Go to the Insight page. > 2.Use F5 to refresh this page. > 3.Click "Add Project" button.(Not "Manage Project" button) > There is a "GET http://10.43.159.142:7070/kylin/project.html 404 (Not Found)" > error in the browser console. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2642) Relax check in RowKeyColDesc to keep backward compatibility
[
https://issues.apache.org/jira/browse/KYLIN-2642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2642:
-
Issue Type: Improvement (was: Bug)
> Relax check in RowKeyColDesc to keep backward compatibility
> ---
>
> Key: KYLIN-2642
> URL: https://issues.apache.org/jira/browse/KYLIN-2642
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>Priority: Minor
> Fix For: v2.1.0
>
> Attachments: KYLIN-2642.patch
>
>
> This check will make the cube DESCBROKEN if user used FixedLenDimEnc encode
> integer:
> {code:java}
> if (encodingName.startsWith(FixedLenDimEnc.ENCODING_NAME) &&
> (type.isIntegerFamily() || type.isNumberFamily())) {
> throw new IllegalArgumentException(colRef + " type is " + type +
> " and cannot apply fixed_length encoding");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2637) Show tips after creating project successfully
[
https://issues.apache.org/jira/browse/KYLIN-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2637:
-
Summary: Show tips after creating project successfully (was: The tips is
not shown after creating project successfully.)
> Show tips after creating project successfully
> -
>
> Key: KYLIN-2637
> URL: https://issues.apache.org/jira/browse/KYLIN-2637
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.1.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>Priority: Trivial
> Fix For: v2.1.0
>
> Attachments:
> 0001-KYLIN-2637-The-tips-is-not-shown-after-creating-proj.patch
>
>
> The following code in page.js:
> SweetAlert.swal('Success!', 'New project created successfully!',
> 'success');
> $modalInstance.dismiss('cancel');
> $cookieStore.put("project", newProj.name);
> location.reload();
> 'SweetAlert' is invoked before 'location.reload()', so it cannot be shown.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2621) Fix issue on mapping LDAP group to the admin group
[ https://issues.apache.org/jira/browse/KYLIN-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2621: - Summary: Fix issue on mapping LDAP group to the admin group (was: An issue about maping an LDAP group to the admin group in Kylin) > Fix issue on mapping LDAP group to the admin group > -- > > Key: KYLIN-2621 > URL: https://issues.apache.org/jira/browse/KYLIN-2621 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2621-The-user-of-the-LDAP-group-named-admin-al.patch > > > The LDAP group named "admin" always has ROLE_ADMIN permission, even if > "kylin.security.acl.admin-role" is set to another group. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2624) Correct reporting of HBase errors
[ https://issues.apache.org/jira/browse/KYLIN-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2624: - Issue Type: Improvement (was: Bug) > Correct reporting of HBase errors > - > > Key: KYLIN-2624 > URL: https://issues.apache.org/jira/browse/KYLIN-2624 > Project: Kylin > Issue Type: Improvement >Reporter: liyang >Assignee: liyang > Fix For: v2.1.0 > > > Whenever HBase error occurs, metadata access fails, user must see a clear > error message saying it is HBase failing, not Kylin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2616) GUI for multiple column count distinct measure
[ https://issues.apache.org/jira/browse/KYLIN-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2616: - Issue Type: Improvement (was: Bug) > GUI for multiple column count distinct measure > -- > > Key: KYLIN-2616 > URL: https://issues.apache.org/jira/browse/KYLIN-2616 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: liyang >Assignee: Zhixiong Chen > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2616-COUNT-DISTINCT-SUPPORT-MULTIPLE-COLUMN.patch > > > Although KYLIN-490 is done, user still cannot create a count distinct measure > with multiple column. The GUI of creating measure needs a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2616) GUI for multiple column count distinct measure
[ https://issues.apache.org/jira/browse/KYLIN-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2616: - Summary: GUI for multiple column count distinct measure (was: Multiple column count distinct misses GUI) > GUI for multiple column count distinct measure > -- > > Key: KYLIN-2616 > URL: https://issues.apache.org/jira/browse/KYLIN-2616 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: liyang >Assignee: Zhixiong Chen > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2616-COUNT-DISTINCT-SUPPORT-MULTIPLE-COLUMN.patch > > > Although KYLIN-490 is done, user still cannot create a count distinct measure > with multiple column. The GUI of creating measure needs a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2613) Wrong variable is used in DimensionDesc#hashCode
[
https://issues.apache.org/jira/browse/KYLIN-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2613:
Assignee: liyang
> Wrong variable is used in DimensionDesc#hashCode
>
>
> Key: KYLIN-2613
> URL: https://issues.apache.org/jira/browse/KYLIN-2613
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liyang
> Fix For: v2.1.0
>
>
> Here is related code:
> {code}
>result = prime * result + ((name == null) ? 0 : name.hashCode());
> result = prime * result + ((column == null) ? 0 : name.hashCode());
> {code}
> The second line is referencing name where column should be referenced.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2609) Fix grant role access issue on project page.
[ https://issues.apache.org/jira/browse/KYLIN-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2609: - Summary: Fix grant role access issue on project page. (was: Grant role access to the project failed on the project manage page.) > Fix grant role access issue on project page. > > > Key: KYLIN-2609 > URL: https://issues.apache.org/jira/browse/KYLIN-2609 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.1.0 >Reporter: peng.jianhua >Assignee: peng.jianhua > Labels: patch > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2609-Grant-role-access-to-the-project-failed-o.patch > > > Grant role access to the project failed on the project manage page and it > prompts "User not found". > The reproducible steps are as follows: > 1.Grant user "mr" access to project "xxx", and it prompts "User mr does not > exist. Please make sure the user has logged in before". > 2.Then I chose the type of role and grant access to "ROLE_ANALYST", it > prompts "User not found". Because the http request URL is missing the > project's uuid. > Finally I refresh the page and re-execute the second step, it grants access > successfully. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2605) Remove the hard-code sample data path in 'sample.sh'
[
https://issues.apache.org/jira/browse/KYLIN-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu updated KYLIN-2605:
-
Summary: Remove the hard-code sample data path in 'sample.sh' (was:
Execute sample.sh script failure in hdfs federation environment)
> Remove the hard-code sample data path in 'sample.sh'
>
>
> Key: KYLIN-2605
> URL: https://issues.apache.org/jira/browse/KYLIN-2605
> Project: Kylin
> Issue Type: Bug
> Components: General
>Affects Versions: v2.0.0
> Environment: HDFS Federation
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>Priority: Minor
> Labels: patch
> Fix For: v2.1.0
>
> Attachments:
> 0001-KYLIN-2605-Execute-sample.sh-script-failure-in-hdfs-.patch
>
>
> Execute sample.sh script failure in hdfs federation environment, because of
> creating hdfs directory failed. The error log is as following:
> Loading sample data into HDFS tmp path: /tmp/kylin/sample_cube/data
> mkdir: /tmp
> mkdir: InternalDir of ViewFileSystem is readonly;
> operation=mkdirsPath=/tmp/kylin/sample_cube/data
> put: `/tmp/kylin/sample_cube/data/': No such file or directory
> In my environment, the hdfs directory should be '/nameservice/tmp/kylin', but
> there is hard coding in sample.sh:
> hadoop ${hadoop_conf_param} fs -mkdir -p /tmp/kylin/sample_cube/data
> hadoop ${hadoop_conf_param} fs -put * /tmp/kylin/sample_cube/data/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2572) Fix parsing 'hive_home' error in 'find-hive-dependency.sh'
[
https://issues.apache.org/jira/browse/KYLIN-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Billy Liu reassigned KYLIN-2572:
Assignee: peng.jianhua (was: Billy Liu)
> Fix parsing 'hive_home' error in 'find-hive-dependency.sh'
> --
>
> Key: KYLIN-2572
> URL: https://issues.apache.org/jira/browse/KYLIN-2572
> Project: Kylin
> Issue Type: Bug
> Components: General
>Affects Versions: v2.0.0
>Reporter: peng.jianhua
>Assignee: peng.jianhua
> Labels: patch
> Fix For: v2.1.0
>
> Attachments:
> 0001-KYLIN-2572-Kylin-failed-to-start-because-of-parsing-.patch
>
>
> Kylin failed to start because of parsing 'hive_home' error in
> 'find-hive-dependency.sh'
> The error log is as following:
> Couldn't locate hcatalog installation, please make sure it is installed and
> set HCAT_HOME to the path.
> In my environment, the HIVE_HOME is /opt/parcels/lib/mr/hive-2.1.0 and the
> path contains '/lib/'
> The code in find-hive-dependency.sh:
> 75 hive_home=`echo $hive_exec_path | awk -F '/lib/' '{print $1}'`
> Using '/lib/' as a delimiter, the result is that getting the wrong hive_home
> : '/opt/parcels'
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2595) Display column alias name when query with keyword 'As'
[ https://issues.apache.org/jira/browse/KYLIN-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2595: - Summary: Display column alias name when query with keyword 'As' (was: Result query should use alias name to replace dimension column name when used Keyword As) > Display column alias name when query with keyword 'As' > -- > > Key: KYLIN-2595 > URL: https://issues.apache.org/jira/browse/KYLIN-2595 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.1.0 >Reporter: wuyingjun >Assignee: wuyingjun > Fix For: v2.1.0 > > Attachments: after_fix.jpg, before_fix.jpg, KYLIN-2595.patch > > > when i check the result, the result found in the dimensions suggests that > the title is column name rather than the alias name -- This message was sent by Atlassian JIRA (v6.4.14#64029)
