[jira] [Resolved] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
[
https://issues.apache.org/jira/browse/KYLIN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3887.
-
Resolution: Fixed
> Query with decimal sum measure of double complied failed after KYLIN-3703
> -
>
> Key: KYLIN-3887
> URL: https://issues.apache.org/jira/browse/KYLIN-3887
> Project: Kylin
> Issue Type: Bug
>Reporter: Shaohui Liu
>Assignee: nichunen
>Priority: Major
> Fix For: Future, v3.0.0
>
> Attachments: image-2019-05-14-11-19-05-514.png,
> image-2019-12-02-13-06-21-282.png
>
>
> After KYLIN-3703, Query with decimal sum measure of double complied failed.
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException:
> Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4260) When using server side PreparedStatement cache, the query result are not match on TopN scenario
[
https://issues.apache.org/jira/browse/KYLIN-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4260:
Fix Version/s: (was: v3.0.0)
Future
> When using server side PreparedStatement cache, the query result are not
> match on TopN scenario
> ---
>
> Key: KYLIN-4260
> URL: https://issues.apache.org/jira/browse/KYLIN-4260
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.0.0-alpha2, v2.6.4
>Reporter: Marc Wu
>Assignee: Marc Wu
>Priority: Major
> Fix For: Future, v2.6.5
>
> Attachments: image-2019-11-18-15-55-00-312.png,
> image-2019-11-18-15-55-11-906.png, image-2019-11-18-19-29-34-489.png,
> image-2019-11-18-19-29-42-721.png
>
>
> Hi Kylin team,
> I found an issue while server side PreparedStatement enabled. The second time
> query and after's result will be different from the first when query TopN,
> and the result is not right.
> Part of Cube info:
>
> Dimensions
> TRANS_ID
> PART_DT
> SELLER_ID
> BUYER_ID
> Measures:
> SUM(PRICE)
> MAX(PRICE)
> TOPN(PRICE) Group By:KYLIN_SALES.SELLER_ID,KYLIN_SALES.BUYER_ID
>
> SQL:
> {code:java}
> {"sql":"select seller_id, buyer_id, sum(PRICE) from glaucus.kylin_sales where
> PART_DT >= ? and PART_DT <= ? group by seller_id, buyer_id order by
> sum(PRICE) desc limit 20","project":"DDTFORTEST_Analytics",
> "params":[{"className": "java.lang.String","value":
> "2012-01-01"},{"className": "java.lang.String","value": "2012-01-10"}]}
> {code}
> The First query result:
> !image-2019-11-18-15-55-00-312.png!
> The Second and after:
> !image-2019-11-18-15-55-11-906.png|width=2046,height=1096!
> ---
> h2. -Root Cause-
> Cached preparedContext is changed when doing preparedStatement.executeQuery,
> and losing groupByColumns. So the first execution result is correct, the
> second and the after will be incorrect.
> !image-2019-11-18-19-29-34-489.png!
> !image-2019-11-18-19-29-42-721.png!
> h2. Real Root Cause
> The first time we entered PreparedStatement logic, we'll try to borrow
> preparedContext from cache pool, of course there isn't any, but the cache
> pool will execute create method to create a new preparedContext, and then
> loaned it to us.
> I didn't figure out how adjustSqlDigest works before, and try to remove code
> {code:java}
> sqlDigest.groupbyColumns.removeAll(topnLiteralCol){code}
> but it's not right. Top-N isn't like some other measures, the dimensions
> aren't as part of row key, they stored in measures in design, so it's why the
> adjustSqlDigest matters, especially those codes.
> {code:java}
> sqlDigest.groupbyColumns.removeAll(topnLiteralCol);
> sqlDigest.metricColumns.addAll(topnLiteralCol);
> {code}
> The root cause for this issue is because the create sql digest is execute
> again after we store it in cache, so the digest changed.
> {code:java}
> # This is from the first time
> fact table GLAUCUS.KYLIN_SALES,group by [],filter on
> [GLAUCUS.KYLIN_SALES.PART_DT],with aggregates[FunctionDesc [expression=TOP_N,
> parameter=GLAUCUS.KYLIN_SALES.PRICE,GLAUCUS.KYLIN_SALES.SELLER_ID,GLAUCUS.KYLIN_SALES.BUYER_ID,
> returnType=topn(5000,8)]].
> # This is the second one
> fact table GLAUCUS.KYLIN_SALES,group by [],filter on
> [GLAUCUS.KYLIN_SALES.PART_DT],with aggregates[FunctionDesc [expression=SUM,
> parameter=GLAUCUS.KYLIN_SALES.PRICE, returnType=decimal(19,4)]].
> {code}
> So the second time and after we execute the same query or same pattern, the
> expression will be changed to SUM instead of TOPN, that's why the strange
> result show up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4187) Building dimension dictionary using spark
[ https://issues.apache.org/jira/browse/KYLIN-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985549#comment-16985549 ] nichunen commented on KYLIN-4187: - [~wangrupeng] PR merged. Would you please add a pr for the config? > Building dimension dictionary using spark > - > > Key: KYLIN-4187 > URL: https://issues.apache.org/jira/browse/KYLIN-4187 > Project: Kylin > Issue Type: Improvement >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Minor > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4187) Building dimension dictionary using spark
[ https://issues.apache.org/jira/browse/KYLIN-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4187. - Resolution: Fixed > Building dimension dictionary using spark > - > > Key: KYLIN-4187 > URL: https://issues.apache.org/jira/browse/KYLIN-4187 > Project: Kylin > Issue Type: Improvement >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Minor > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4229) String index out of range -1
[
https://issues.apache.org/jira/browse/KYLIN-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4229.
-
Resolution: Fixed
> String index out of range -1
>
>
> Key: KYLIN-4229
> URL: https://issues.apache.org/jira/browse/KYLIN-4229
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v3.0.0-beta
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0
>
>
> Build cube with "where clause to filter data from source" will get the error
> "String index out of range -1"
> {code:java}
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.StringBuffer.setCharAt(StringBuffer.java:255)
> at
> org.apache.commons.lang3.time.FastDatePrinter$PaddedNumberField.appendTo(FastDatePrinter.java:885)
> at
> org.apache.commons.lang3.time.FastDatePrinter$PaddedNumberField.appendTo(FastDatePrinter.java:870)
> at
> org.apache.commons.lang3.time.FastDatePrinter.applyRules(FastDatePrinter.java:493)
> at
> org.apache.commons.lang3.time.FastDatePrinter.applyRulesToString(FastDatePrinter.java:427)
> at
> org.apache.commons.lang3.time.FastDatePrinter.format(FastDatePrinter.java:446)
> at
> org.apache.commons.lang3.time.FastDateFormat.format(FastDateFormat.java:434)
> at
> org.apache.kylin.common.util.DateFormat.formatToDateStr(DateFormat.java:69)
> at
> org.apache.kylin.job.JoinedFormatter.setDateEnv(JoinedFormatter.java:74)
> at
> org.apache.kylin.job.JoinedFormatter.(JoinedFormatter.java:56)
> at
> org.apache.kylin.job.JoinedFlatTable.appendWhereStatement(JoinedFlatTable.java:228)
> at
> org.apache.kylin.job.JoinedFlatTable.generateSelectDataStatement(JoinedFlatTable.java:164)
> at
> org.apache.kylin.job.JoinedFlatTable.generateSelectDataStatement(JoinedFlatTable.java:136)
> at
> org.apache.kylin.job.JoinedFlatTable.generateInsertDataStatement(JoinedFlatTable.java:131)
> at
> org.apache.kylin.source.hive.HiveInputBase.createFlatHiveTableStep(HiveInputBase.java:319)
> at
> org.apache.kylin.source.hive.HiveInputBase$BaseBatchCubingInputSide.addStepPhase1_DoCreateFlatTable(HiveInputBase.java:259)
> at
> org.apache.kylin.source.hive.HiveInputBase$BaseBatchCubingInputSide.addStepPhase1_CreateFlatTable(HiveInputBase.java:89)
> at
> org.apache.kylin.engine.mr.BatchCubingJobBuilder2.build(BatchCubingJobBuilder2.java:59)
> at
> org.apache.kylin.engine.mr.MRBatchCubingEngine2.createBatchCubingJob(MRBatchCubingEngine2.java:42)
> at
> org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:61)
> at
> org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:247)
> at
> org.apache.kylin.rest.service.JobService.submitJob(JobService.java:215)
> at
> org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:402)
> at
> org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:362)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:653)
> at
>
[jira] [Resolved] (KYLIN-4190) hiveproducer write() function throw exception because hive mertics table location path prefix is different with defaut fs when hdfs uses router-based federation
[
https://issues.apache.org/jira/browse/KYLIN-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4190.
-
Resolution: Fixed
> hiveproducer write() function throw exception because hive mertics table
> location path prefix is different with defaut fs when hdfs uses router-based
> federation
>
>
> Key: KYLIN-4190
> URL: https://issues.apache.org/jira/browse/KYLIN-4190
> Project: Kylin
> Issue Type: Bug
> Components: Metrics
>Affects Versions: v3.0.0-alpha
>Reporter: chuxiao
>Priority: Critical
> Fix For: v3.0.0
>
> Attachments: KYLIN-4190.master.001.patch
>
>
> Our hdfs cluster uses router-based
> federation（[https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html]).
> Opening the system cube configuration, hiveproducer write() function throw
> exception :
> {code:java}
> //代码占位符
> ERROR [metrics-blocking-reservoir-scheduler-0] hive.HiveReservoirReporter:119
> : Wrong FS:
> hdfs://DClusterNmg4/user/kylin/hive/hive_metrics_job_exception_qa/kday_date=2019-09-04,
> expected: hdfs://difed
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://DClusterNmg4/user/kylin/hive/hive_metrics_job_exception_qa/kday_date=2019-09-04,
> expected: hdfs://difed
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:717)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:197)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1390)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1386)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1402)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1494)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.write(HiveProducer.java:137)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.send(HiveProducer.java:122)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter$HiveReservoirListener.onRecordUpdate(HiveReservoirReporter.java:117)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.notifyListenerOfUpdatedRecord(BlockingReservoir.java:105)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.onRecordUpdate(BlockingReservoir.java:93)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.access$300(BlockingReservoir.java:33)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir$ReporterRunnable.run(BlockingReservoir.java:152)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
> This is because the default router namespace is hdfs://difed, and the actual
> federation namespaces are the hdfs://DClusterNmg4, the hdfs://DClusterNmg1,
> and the hdfs://DClusterNmg2...
> So fs.defaultFS in core-sie.xml is hdfs ://difed, But this hive table
> location path is hdfs://DClusterNmg4/user/... . Then
> defaultFs.exists(hiveLocationPath) throw exception.
> So we need to check if the prefix is same. If defaut fs is not a prefix of
> hive table location path, use location path get a new filesystem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4203) Disable a real time cube and then enable it ,this cube may can't submit build job anymore
[ https://issues.apache.org/jira/browse/KYLIN-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4203. - Resolution: Fixed > Disable a real time cube and then enable it ,this cube may can't submit build > job anymore > - > > Key: KYLIN-4203 > URL: https://issues.apache.org/jira/browse/KYLIN-4203 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: Future >Reporter: wangxiaojing >Assignee: wangxiaojing >Priority: Blocker > Fix For: v3.0.0 > > Attachments: image-2019-10-25-18-35-51-570.png > > > First ,disable a real time streaming cube when the cube has max building jobs > (default max job size is 10),then enable the cube。But this cube may can't > subbmit new building jobs any more even if the kylin user have discarded the > building jobs ,it logs "No left quota to build segments for cube". Because > the amount of left quota one cube can submit building jobs is determined by > this algorithm: allowMaxBuildingSegments - inBuildingSegments". The > 'allowMaxBuildingSegments' is configed and the 'inBuildingSegments ' are the > cube's not ready segments in hbase(perhaps some other storage). > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4203) Disable a real time cube and then enable it ,this cube may can't submit build job anymore
[ https://issues.apache.org/jira/browse/KYLIN-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4203: Fix Version/s: (was: Future) v3.0.0 > Disable a real time cube and then enable it ,this cube may can't submit build > job anymore > - > > Key: KYLIN-4203 > URL: https://issues.apache.org/jira/browse/KYLIN-4203 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: Future >Reporter: wangxiaojing >Assignee: wangxiaojing >Priority: Blocker > Fix For: v3.0.0 > > Attachments: image-2019-10-25-18-35-51-570.png > > > First ,disable a real time streaming cube when the cube has max building jobs > (default max job size is 10),then enable the cube。But this cube may can't > subbmit new building jobs any more even if the kylin user have discarded the > building jobs ,it logs "No left quota to build segments for cube". Because > the amount of left quota one cube can submit building jobs is determined by > this algorithm: allowMaxBuildingSegments - inBuildingSegments". The > 'allowMaxBuildingSegments' is configed and the 'inBuildingSegments ' are the > cube's not ready segments in hbase(perhaps some other storage). > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4260) When using server side PreparedStatement cache, the query result are not match on TopN scenario
[
https://issues.apache.org/jira/browse/KYLIN-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984346#comment-16984346
]
nichunen commented on KYLIN-4260:
-
I have reproduced the bug, it seems that it's introduced by KYLIN-3434.
> When using server side PreparedStatement cache, the query result are not
> match on TopN scenario
> ---
>
> Key: KYLIN-4260
> URL: https://issues.apache.org/jira/browse/KYLIN-4260
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.0.0-alpha2, v2.6.4
>Reporter: Marc Wu
>Assignee: Marc Wu
>Priority: Major
> Fix For: v3.0.0, v2.6.5
>
> Attachments: image-2019-11-18-15-55-00-312.png,
> image-2019-11-18-15-55-11-906.png, image-2019-11-18-19-29-34-489.png,
> image-2019-11-18-19-29-42-721.png
>
>
> Hi Kylin team,
> I found an issue while server side PreparedStatement enabled. The second time
> query and after's result will be different from the first when query TopN,
> and the result is not right.
> Part of Cube info:
>
> Dimensions
> TRANS_ID
> PART_DT
> SELLER_ID
> BUYER_ID
> Measures:
> SUM(PRICE)
> MAX(PRICE)
> TOPN(PRICE) Group By:KYLIN_SALES.SELLER_ID,KYLIN_SALES.BUYER_ID
>
> SQL:
> {code:java}
> {"sql":"select seller_id, buyer_id, sum(PRICE) from glaucus.kylin_sales where
> PART_DT >= ? and PART_DT <= ? group by seller_id, buyer_id order by
> sum(PRICE) desc limit 20","project":"DDTFORTEST_Analytics",
> "params":[{"className": "java.lang.String","value":
> "2012-01-01"},{"className": "java.lang.String","value": "2012-01-10"}]}
> {code}
> The First query result:
> !image-2019-11-18-15-55-00-312.png!
> The Second and after:
> !image-2019-11-18-15-55-11-906.png|width=2046,height=1096!
> ---
> h2. Root Cause
> Cached preparedContext is changed when doing preparedStatement.executeQuery,
> and losing groupByColumns. So the first execution result is correct, the
> second and the after will be incorrect.
> !image-2019-11-18-19-29-34-489.png!
> !image-2019-11-18-19-29-42-721.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4257) Build historical data by layer in real time Lambda cube
[ https://issues.apache.org/jira/browse/KYLIN-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4257. - Resolution: Fixed > Build historical data by layer in real time Lambda cube > --- > > Key: KYLIN-4257 > URL: https://issues.apache.org/jira/browse/KYLIN-4257 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0 > > > When user create a Lambda cube with a large mount of historical data, build > in memory is too slowly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4254) The result exporting from Insight with CSV format is empty, when sql contains Chinese
[ https://issues.apache.org/jira/browse/KYLIN-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4254. - Resolution: Fixed > The result exporting from Insight with CSV format is empty, when sql contains > Chinese > - > > Key: KYLIN-4254 > URL: https://issues.apache.org/jira/browse/KYLIN-4254 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.5.3, v2.6.0 >Reporter: Zheng Yang >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > > When the query results of the Insight page are exported, sql is passed as a > parameter to the URL of the exported data, which causes the problem of > garbled Chinese parameters in the URL. > The encoding used by Tomcat for default decoding is the container's default > encoding, which may be UTF-8, GBK, or other encoding. This is not necessarily > the same as the encoding of your app. Fix the problem by modifying the > container's default encoding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3212) Replace "array[]" with "array()" when pushing down query to Hive
[ https://issues.apache.org/jira/browse/KYLIN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3212: Fix Version/s: (was: v3.0.0) Future > Replace "array[]" with "array()" when pushing down query to Hive > > > Key: KYLIN-3212 > URL: https://issues.apache.org/jira/browse/KYLIN-3212 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Reporter: Chuqian Yu >Assignee: luguosheng >Priority: Minor > Fix For: Future > > > Hi , Kylin community. > I am using "intersect_count" to calculate the conversion rate and want to > pushing it down to hive when no cube can answer. But the array format > "array[]" is not readable in hive .So I need to add a method in > org.apache.kylin.source.adhocquery.HivePushDownConverter in order to replace > "array[]" to "array()" when pushing down query to Hive. > Any suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3212) Replace "array[]" with "array()" when pushing down query to Hive
[ https://issues.apache.org/jira/browse/KYLIN-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983283#comment-16983283 ] nichunen commented on KYLIN-3212: - [~yuchuqian] Hi, what's your idea for calculating the conversion rate with Hive? As I know, Hive does not support "intersect_count" > Replace "array[]" with "array()" when pushing down query to Hive > > > Key: KYLIN-3212 > URL: https://issues.apache.org/jira/browse/KYLIN-3212 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Reporter: Chuqian Yu >Assignee: luguosheng >Priority: Minor > Fix For: v3.0.0 > > > Hi , Kylin community. > I am using "intersect_count" to calculate the conversion rate and want to > pushing it down to hive when no cube can answer. But the array format > "array[]" is not readable in hive .So I need to add a method in > org.apache.kylin.source.adhocquery.HivePushDownConverter in order to replace > "array[]" to "array()" when pushing down query to Hive. > Any suggestions? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4190) hiveproducer write() function throw exception because hive mertics table location path prefix is different with defaut fs when hdfs uses router-based federation
[
https://issues.apache.org/jira/browse/KYLIN-4190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4190:
Fix Version/s: v3.0.0
> hiveproducer write() function throw exception because hive mertics table
> location path prefix is different with defaut fs when hdfs uses router-based
> federation
>
>
> Key: KYLIN-4190
> URL: https://issues.apache.org/jira/browse/KYLIN-4190
> Project: Kylin
> Issue Type: Bug
> Components: Metrics
>Affects Versions: v3.0.0-alpha
>Reporter: chuxiao
>Priority: Critical
> Fix For: v3.0.0
>
> Attachments: KYLIN-4190.master.001.patch
>
>
> Our hdfs cluster uses router-based
> federation（[https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html]).
> Opening the system cube configuration, hiveproducer write() function throw
> exception :
> {code:java}
> //代码占位符
> ERROR [metrics-blocking-reservoir-scheduler-0] hive.HiveReservoirReporter:119
> : Wrong FS:
> hdfs://DClusterNmg4/user/kylin/hive/hive_metrics_job_exception_qa/kday_date=2019-09-04,
> expected: hdfs://difed
> java.lang.IllegalArgumentException: Wrong FS:
> hdfs://DClusterNmg4/user/kylin/hive/hive_metrics_job_exception_qa/kday_date=2019-09-04,
> expected: hdfs://difed
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:717)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:197)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1390)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1386)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1402)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1494)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.write(HiveProducer.java:137)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveProducer.send(HiveProducer.java:122)
> at
> org.apache.kylin.metrics.lib.impl.hive.HiveReservoirReporter$HiveReservoirListener.onRecordUpdate(HiveReservoirReporter.java:117)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.notifyListenerOfUpdatedRecord(BlockingReservoir.java:105)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.onRecordUpdate(BlockingReservoir.java:93)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir.access$300(BlockingReservoir.java:33)
> at
> org.apache.kylin.metrics.lib.impl.BlockingReservoir$ReporterRunnable.run(BlockingReservoir.java:152)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
> This is because the default router namespace is hdfs://difed, and the actual
> federation namespaces are the hdfs://DClusterNmg4, the hdfs://DClusterNmg1,
> and the hdfs://DClusterNmg2...
> So fs.defaultFS in core-sie.xml is hdfs ://difed, But this hive table
> location path is hdfs://DClusterNmg4/user/... . Then
> defaultFs.exists(hiveLocationPath) throw exception.
> So we need to check if the prefix is same. If defaut fs is not a prefix of
> hive table location path, use location path get a new filesystem
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3881) Calcite isolating expression with its condition may throw 'Division Undefined' exception
[
https://issues.apache.org/jira/browse/KYLIN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3881.
-
Resolution: Fixed
> Calcite isolating expression with its condition may throw 'Division
> Undefined' exception
>
>
> Key: KYLIN-3881
> URL: https://issues.apache.org/jira/browse/KYLIN-3881
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
> Attachments: APACHE-KYLIN-3881.patch
>
>
> When invoke the following sql:
> {code:java}
> SELECT LSTG_FORMAT_NAME, (case when sum(PRICE) <> 0 then
> sum(PRICE)/min(price) else 0 end)
> FROM TEST_KYLIN_FACT
> GROUP BY LSTG_FORMAT_NAME
> HAVING (case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end >= 0.5
> and case when sum(PRICE) <> 0 then sum(PRICE)/min(price) else 0 end
> <= 0.75)
> {code}
> The generated code of moveMext part is as follow:
> {code:java}
> /* 157 */ public boolean moveNext() {
> /* 158 */ while (inputEnumerator.moveNext()) {
> /* 159 */ final Object[] current = (Object[]) inputEnumerator.current();
> /* 160 */ final java.math.BigDecimal inp1_ = current[1] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[1]);
> /* 161 */ final java.math.BigDecimal inp2_ = current[2] == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[2]);
> /* 162 */ final java.math.BigDecimal v0 =
> org.apache.calcite.runtime.SqlFunctions.divide(inp1_, inp2_);
> /* 163 */ if (inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_) && (inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ge(v0,
> $L4J$C$new_java_math_BigDecimal_0_5_)) && (!(inp1_ != null &&
> org.apache.calcite.runtime.SqlFunctions.ne(inp1_,
> $L4J$C$new_java_math_BigDecimal_0_)) || inp1_ != null && inp2_ != null &&
> org.apache.calcite.runtime.SqlFunctions.le(v0,
> $L4J$C$new_java_math_BigDecimal_0_75_))) {
> /* 164 */ return true;
> /* 165 */ }
> /* 166 */ }
> /* 167 */ return false;
> /* 168 */ }
> {code}
> line 162, the divide logic should be combined with its condition. Otherwise,
> the 'Division Undefined' exception will happen.
> The whole generated code is as follows:
> {code:java}
> /* 1 */ public static class Record4_0 implements java.io.Serializable {
> /* 2 */ public java.math.BigDecimal f0;
> /* 3 */ public boolean f1;
> /* 4 */ public java.math.BigDecimal f2;
> /* 5 */ public boolean f3;
> /* 6 */ public Record4_0() {}
> /* 7 */ public boolean equals(Object o) {
> /* 8 */ if (this == o) {
> /* 9 */ return true;
> /* 10 */ }
> /* 11 */ if (!(o instanceof Record4_0)) {
> /* 12 */ return false;
> /* 13 */ }
> /* 14 */ return java.util.Objects.equals(this.f0, ((Record4_0) o).f0) &&
> this.f1 == ((Record4_0) o).f1 && java.util.Objects.equals(this.f2,
> ((Record4_0) o).f2) && this.f3 == ((Record4_0) o).f3;
> /* 15 */ }
> /* 16 */
> /* 17 */ public int hashCode() {
> /* 18 */ int h = 0;
> /* 19 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
> /* 20 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
> /* 21 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f2);
> /* 22 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f3);
> /* 23 */ return h;
> /* 24 */ }
> /* 25 */
> /* 26 */ public int compareTo(Record4_0 that) {
> /* 27 */ int c;
> /* 28 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f0, that.f0);
> /* 29 */ if (c != 0) {
> /* 30 */ return c;
> /* 31 */ }
> /* 32 */ c = org.apache.calcite.runtime.Utilities.compare(this.f1,
> that.f1);
> /* 33 */ if (c != 0) {
> /* 34 */ return c;
> /* 35 */ }
> /* 36 */ c =
> org.apache.calcite.runtime.Utilities.compareNullsLast(this.f2, that.f2);
> /* 37 */ if (c != 0) {
> /* 38 */ return c;
> /* 39 */ }
> /* 40 */ c = org.apache.calcite.runtime.Utilities.compare(this.f3,
> that.f3);
> /* 41 */ if (c != 0) {
> /* 42 */ return c;
> /* 43 */ }
> /* 44 */ return 0;
> /* 45 */ }
> /* 46 */
> /* 47 */ public String toString() {
> /* 48 */ return "{f0=" + this.f0 + ", f1=" + this.f1 + ", f2=" + this.f2
> + ", f3=" + this.f3 + "}";
> /* 49 */ }
> /* 50 */
> /* 51 */ }
> /* 52 */
> /* 53 */ org.apache.calcite.DataContext root;
> /* 54 */
> /* 55 */ public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root0) {
> /* 56 */ root = root0;
> /* 57 */ final
[jira] [Commented] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982278#comment-16982278
]
nichunen commented on KYLIN-3842:
-
[~seva_ostapenko] Hi, the patch you submitted has been merged to master, thanks
for your contribution
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: Future, v3.0.0
>
> Attachments: KYLIN-3842.master.001.patch
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3842.
-
Resolution: Fixed
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: Future, v3.0.0
>
> Attachments: KYLIN-3842.master.001.patch
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4160) Auto redirect to host:port/kylin when user only enter host:port in broswer
[ https://issues.apache.org/jira/browse/KYLIN-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4160: Fix Version/s: v2.6.5 > Auto redirect to host:port/kylin when user only enter host:port in broswer > -- > > Key: KYLIN-4160 > URL: https://issues.apache.org/jira/browse/KYLIN-4160 > Project: Kylin > Issue Type: Improvement >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.5 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4135) Real time streaming segment build task discard but can't be rebuilt
[ https://issues.apache.org/jira/browse/KYLIN-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4135. - Resolution: Fixed > Real time streaming segment build task discard but can't be rebuilt > > > Key: KYLIN-4135 > URL: https://issues.apache.org/jira/browse/KYLIN-4135 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: Future >Reporter: wangxiaojing >Priority: Major > Fix For: v3.0.0 > > > Real time streaming segment build task discard in some case ,but now can't be > rebuilt, and cant't subbmit other new segment tsrange to build if it reach > the cube's max building number( > [https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-4134?filter=allopenissues]) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4162) After drop the build task on the monitor page, subsequent segments cannot be constructed.
[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4162. - Resolution: Fixed > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > - > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4261) Synchronization performed on java.util.concurrent.ConcurrentMap in AssignmentsCache
[ https://issues.apache.org/jira/browse/KYLIN-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4261: Fix Version/s: (was: v3.0.0) Future > Synchronization performed on java.util.concurrent.ConcurrentMap in > AssignmentsCache > --- > > Key: KYLIN-4261 > URL: https://issues.apache.org/jira/browse/KYLIN-4261 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2, v2.6.4 >Reporter: Dezhi Cai >Priority: Major > Fix For: Future > > > FindBugs report issue "Synchronization performed on > java.util.concurrent.ConcurrentMap" > in class org.apache.kylin.stream.coordinator.assign.AssignmentsCache -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4172) Can't rename field when map streaming schema to table
[ https://issues.apache.org/jira/browse/KYLIN-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4172. - Resolution: Fixed > Can't rename field when map streaming schema to table > - > > Key: KYLIN-4172 > URL: https://issues.apache.org/jira/browse/KYLIN-4172 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Peng Huang >Priority: Major > Fix For: v3.0.0 > > Attachments: 微信截图_20190919102424.png > > > When I map streaming schema to table, I don't know how to do it by myself. I > have to use auto-mapping by which i can't rename field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4258) Real-time OLAP may return incorrect result for some case
[ https://issues.apache.org/jira/browse/KYLIN-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4258: Fix Version/s: v3.0.0 > Real-time OLAP may return incorrect result for some case > > > Key: KYLIN-4258 > URL: https://issues.apache.org/jira/browse/KYLIN-4258 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2 >Reporter: Gang Ma >Assignee: Gang Ma >Priority: Major > Fix For: v3.0.0 > > > 1. When real-time query group by minute_start and filter by another column, > the aggregate result maybe not correct, the issue has been addressed in > [KYLIN-4184|https://issues.apache.org/jira/browse/KYLIN-4184] , the fix is > good, but in > NoCompressedColumnReader class, the readBuffer bytes is still shared, it is a > potential issue, need to be fixed. > 2. When filter by a minute_start range, the first minute data may not be > correct, it is caused > by:[https://github.com/apache/kylin/blob/master/stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarSegmentStoreFilesSearcher.java#L68], > the fragment's max time is inclusive, but this method's max time is > exclusive, that may skip some fragments for the first minute by mistake. > 3. Need to add some UT to cover these two cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4178) Job scheduler support safe mode
[ https://issues.apache.org/jira/browse/KYLIN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4178. - Resolution: Fixed > Job scheduler support safe mode > > > Key: KYLIN-4178 > URL: https://issues.apache.org/jira/browse/KYLIN-4178 > Project: Kylin > Issue Type: Improvement >Reporter: ZhouKang >Priority: Major > Fix For: v3.0.0 > > > Job scheduler should support safe mode in case of the HBase cluster change. > In xiaomi, we want update the HBase cluster from hbase0.98 to HBase 2.0. The > history data can be migrated previously, but the job has been submitted will > keep running and write data to the old cluster. So we need a method to ensure > that, job create htable in the old cluster will write data to the old > cluster, and the job have not create htable should not be scheduled. > So we need job scheduler safe mode. Open safe mode before changing cluster > config, the running jobs can run continuous, and the new job cannot be > scheduled. > After all running job finished, we can change the cluster config to the new > one, and rest of job can be scheduled again. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4193) More user-friendly page for loading streaming tables
[ https://issues.apache.org/jira/browse/KYLIN-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4193. - Resolution: Fixed > More user-friendly page for loading streaming tables > > > Key: KYLIN-4193 > URL: https://issues.apache.org/jira/browse/KYLIN-4193 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha >Reporter: nichunen >Priority: Major > Fix For: v3.0.0 > > > After click "Add Streaming Table V2", the user has to set "TSParser" and > "TSPattern", these items may confuse users, they should be made more > user-friendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4172) Can't rename field when map streaming schema to table
[ https://issues.apache.org/jira/browse/KYLIN-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4172: Fix Version/s: (was: Future) v3.0.0 > Can't rename field when map streaming schema to table > - > > Key: KYLIN-4172 > URL: https://issues.apache.org/jira/browse/KYLIN-4172 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Peng Huang >Priority: Major > Fix For: v3.0.0 > > Attachments: 微信截图_20190919102424.png > > > When I map streaming schema to table, I don't know how to do it by myself. I > have to use auto-mapping by which i can't rename field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4201) Allow users to delete unused receivers from streaming page
[ https://issues.apache.org/jira/browse/KYLIN-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4201: Fix Version/s: (was: Future) v3.0.0 > Allow users to delete unused receivers from streaming page > -- > > Key: KYLIN-4201 > URL: https://issues.apache.org/jira/browse/KYLIN-4201 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0 > > Attachments: image-2019-10-30-17-14-51-540.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4201) Allow users to delete unused receivers from streaming page
[ https://issues.apache.org/jira/browse/KYLIN-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4201. - Resolution: Fixed > Allow users to delete unused receivers from streaming page > -- > > Key: KYLIN-4201 > URL: https://issues.apache.org/jira/browse/KYLIN-4201 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: Future > > Attachments: image-2019-10-30-17-14-51-540.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4254) The result exporting from Insight with CSV format is empty, when sql contains Chinese
[ https://issues.apache.org/jira/browse/KYLIN-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4254: Fix Version/s: (was: v3.0.0-beta) v3.0.0 > The result exporting from Insight with CSV format is empty, when sql contains > Chinese > - > > Key: KYLIN-4254 > URL: https://issues.apache.org/jira/browse/KYLIN-4254 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.5.3, v2.6.0 >Reporter: Zheng Yang >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > > When the query results of the Insight page are exported, sql is passed as a > parameter to the URL of the exported data, which causes the problem of > garbled Chinese parameters in the URL. > The encoding used by Tomcat for default decoding is the container's default > encoding, which may be UTF-8, GBK, or other encoding. This is not necessarily > the same as the encoding of your app. Fix the problem by modifying the > container's default encoding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4262) pid in GC filename inconsistent with real pid
[ https://issues.apache.org/jira/browse/KYLIN-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4262. - Resolution: Fixed > pid in GC filename inconsistent with real pid > - > > Key: KYLIN-4262 > URL: https://issues.apache.org/jira/browse/KYLIN-4262 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0, v2.6.5 > > Attachments: image-2019-11-18-17-19-49-059.png, > image-2019-11-18-17-19-56-990.png, image-2019-11-19-18-55-18-113.png > > > pid in GC filename > !image-2019-11-18-17-19-49-059.png! > > real pid > !image-2019-11-18-17-19-56-990.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4261) Synchronization performed on java.util.concurrent.ConcurrentMap in AssignmentsCache
[ https://issues.apache.org/jira/browse/KYLIN-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4261: Fix Version/s: v3.0.0 > Synchronization performed on java.util.concurrent.ConcurrentMap in > AssignmentsCache > --- > > Key: KYLIN-4261 > URL: https://issues.apache.org/jira/browse/KYLIN-4261 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha, v3.0.0-alpha2, v2.6.4 >Reporter: Dezhi Cai >Priority: Major > Fix For: v3.0.0 > > > FindBugs report issue "Synchronization performed on > java.util.concurrent.ConcurrentMap" > in class org.apache.kylin.stream.coordinator.assign.AssignmentsCache -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4172) Can't rename field when map streaming schema to table
[ https://issues.apache.org/jira/browse/KYLIN-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4172: Fix Version/s: Future > Can't rename field when map streaming schema to table > - > > Key: KYLIN-4172 > URL: https://issues.apache.org/jira/browse/KYLIN-4172 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Peng Huang >Priority: Major > Fix For: Future > > Attachments: 微信截图_20190919102424.png > > > When I map streaming schema to table, I don't know how to do it by myself. I > have to use auto-mapping by which i can't rename field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4257) Build historical data by layer in real time Lambda cube
[ https://issues.apache.org/jira/browse/KYLIN-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4257: Fix Version/s: (was: Future) v3.0.0 > Build historical data by layer in real time Lambda cube > --- > > Key: KYLIN-4257 > URL: https://issues.apache.org/jira/browse/KYLIN-4257 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0 > > > When user create a Lambda cube with a large mount of historical data, build > in memory is too slowly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4201) Allow users to delete unused receivers from streaming page
[ https://issues.apache.org/jira/browse/KYLIN-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4201: - > Allow users to delete unused receivers from streaming page > -- > > Key: KYLIN-4201 > URL: https://issues.apache.org/jira/browse/KYLIN-4201 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: Future > > Attachments: image-2019-10-30-17-14-51-540.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4167) Refactor streaming coordinator
[ https://issues.apache.org/jira/browse/KYLIN-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4167: Fix Version/s: (was: v3.0.0-beta) v3.0.0 > Refactor streaming coordinator > -- > > Key: KYLIN-4167 > URL: https://issues.apache.org/jira/browse/KYLIN-4167 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0 > > > h2. Summary > # Currently, *coordinator* has too many responsibility, which violate single > responsibility principle, and it not easy for extension, a good separation of > responsibilities is a recommended way. > # Some cluster level operation has no atomicity guarantee, we should > implement then in idempotent way to achieve final consistency > # Resubmit when job was discarded > # Clarify overall design for realtime OLAP > > h4. StreamingCoordinator > Facade of coordinator, will controll BuildJobSummitter/ReceiverClusterMangaer > and delegate operation to them. > h4. BuildJobSubmitter > The main responsibility of BuildJobSubmitter including: > 1. Try to find candidate segment which ready to submit a build job > 2. Trace the status of candidate segment's build job and promote segment if > it is has met requirements > h4. > h4. ReceiverClusterManager > This class manage operation related to multi streaming receivers. They are > often not atomic and maybe idempotent. > h4. ClusterStateChecker > Basic step of this class: > 1. stop/pause coordinator to avoid underlying concurrency issue > 2. check inconsistent state of all receiver cluster > 3. send summary via mail to kylin admin > 4. if need, call ClusterDoctor to repair inconsistent issue > h4. ClusterDoctor > Repair inconsistent state according to result of ClusterStateChecker > > > h3. Candidate Segment > The candidate segments are those segments what can be saw/perceived by > streaming coordinator, > candidate segment could be divided into following state/queue: > 1. segment which data are uploaded *PARTLY* > 2. segment which data are uploaded completely and *WAITING* to build > 3. segment which in *BUILDING* state, job's state should be one of > (NEW/RUNNING/ERROR/DISCARD) > 4. segment which built *succeed* and wait to be delivered to historical part > (and to be deleted in realtime part) > 5. segment which *in historical part*(HBase Ready Segment) > > By design, segment should transfer to next queue in sequential way(shouldn't > jump the queue), do not break this. > h3. Atomicity > In a multi-step transcation, following acepts should be thought twice: > 1. should *fail fast* or continue when exception thrown. > 2. should API(remote call) be *synchronous* or asynchronous > 3. when transcation failed, could *roll back* always succeed > 4. transcation should be *idempotent* so when it failed, it could be fixed by > retry > > How to ensure whole cluster opreation smoothly without blocking problem. I > divided all multi-step transcation into three kinds: > NotAtomicIdempotent > NotAtomicAndNotIdempotent > NonSideEffect -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4167) Refactor streaming coordinator
[ https://issues.apache.org/jira/browse/KYLIN-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4167: - > Refactor streaming coordinator > -- > > Key: KYLIN-4167 > URL: https://issues.apache.org/jira/browse/KYLIN-4167 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0 > > > h2. Summary > # Currently, *coordinator* has too many responsibility, which violate single > responsibility principle, and it not easy for extension, a good separation of > responsibilities is a recommended way. > # Some cluster level operation has no atomicity guarantee, we should > implement then in idempotent way to achieve final consistency > # Resubmit when job was discarded > # Clarify overall design for realtime OLAP > > h4. StreamingCoordinator > Facade of coordinator, will controll BuildJobSummitter/ReceiverClusterMangaer > and delegate operation to them. > h4. BuildJobSubmitter > The main responsibility of BuildJobSubmitter including: > 1. Try to find candidate segment which ready to submit a build job > 2. Trace the status of candidate segment's build job and promote segment if > it is has met requirements > h4. > h4. ReceiverClusterManager > This class manage operation related to multi streaming receivers. They are > often not atomic and maybe idempotent. > h4. ClusterStateChecker > Basic step of this class: > 1. stop/pause coordinator to avoid underlying concurrency issue > 2. check inconsistent state of all receiver cluster > 3. send summary via mail to kylin admin > 4. if need, call ClusterDoctor to repair inconsistent issue > h4. ClusterDoctor > Repair inconsistent state according to result of ClusterStateChecker > > > h3. Candidate Segment > The candidate segments are those segments what can be saw/perceived by > streaming coordinator, > candidate segment could be divided into following state/queue: > 1. segment which data are uploaded *PARTLY* > 2. segment which data are uploaded completely and *WAITING* to build > 3. segment which in *BUILDING* state, job's state should be one of > (NEW/RUNNING/ERROR/DISCARD) > 4. segment which built *succeed* and wait to be delivered to historical part > (and to be deleted in realtime part) > 5. segment which *in historical part*(HBase Ready Segment) > > By design, segment should transfer to next queue in sequential way(shouldn't > jump the queue), do not break this. > h3. Atomicity > In a multi-step transcation, following acepts should be thought twice: > 1. should *fail fast* or continue when exception thrown. > 2. should API(remote call) be *synchronous* or asynchronous > 3. when transcation failed, could *roll back* always succeed > 4. transcation should be *idempotent* so when it failed, it could be fixed by > retry > > How to ensure whole cluster opreation smoothly without blocking problem. I > divided all multi-step transcation into three kinds: > NotAtomicIdempotent > NotAtomicAndNotIdempotent > NonSideEffect -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4010) Auto adjust offset according to query server's timezone for time derived column
[
https://issues.apache.org/jira/browse/KYLIN-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4010:
Fix Version/s: (was: v3.0.0-beta)
v3.0.0
> Auto adjust offset according to query server's timezone for time derived
> column
> ---
>
> Key: KYLIN-4010
> URL: https://issues.apache.org/jira/browse/KYLIN-4010
> Project: Kylin
> Issue Type: Improvement
> Components: Others
>Affects Versions: v3.0.0-alpha
>Reporter: zengrui
>Assignee: Xiaoxiang Yu
>Priority: Minor
> Fix For: v3.0.0
>
> Attachments: image-2019-07-15-17-15-31-209.png,
> image-2019-07-15-17-17-04-029.png, image-2019-07-15-17-17-39-568.png,
> image-2019-09-22-16-35-23-663.png, image-2019-09-22-16-43-19-248.png,
> image-2019-09-22-20-28-02-402.png, image-2019-09-22-20-35-19-038.png,
> image-2019-09-22-20-53-34-720.png, image-2019-09-24-17-23-42-451.png,
> image-2019-09-24-17-33-29-824.png, image-2019-09-24-17-35-47-589.png,
> image-2019-09-24-17-36-39-611.png
>
>
> h2. Backgroud
> In realtime OLAP, we index real-time event in streaming receiver. We know
> that each event must contains a timestamp column (we often call it event
> time), that value should represent when this event was produced. Because
> event maybe come from different timezone and use local timezone is always
> *error-prone*, so we recommend to use a {color:#DE350B}GMT+0{color}
> timestamp(System.currentTimeMillis()) to avoid such issue.
> I think this is good by design, it is easy to understand and always correct.
> But the *side effect* is that, the end user(business manager behind a BI
> tools) are unhappy because he have to use GMT+0 with date/time related filter
> in SQL and should understand the result should be *shifted* with his local
> timezone. It is not user-firendly and inconvenient for normal user. Because
> user may compare query result from different data source and compare them and
> summarize, use GMT+0 may trouble them.
> h2. Example
> For example, kylin user work in *GMT+8* (maybe in Shanghai) want to know some
> metrics which occured from {color:#DE350B}2019-09-01 12:00:00{color} to
> {color:#DE350B}2019-09-01 14:00:00{color} in his {color:#DE350B}local
> timezone{color}, so he has to {color:#DE350B}rewrite{color} his query (with
> eight hour offset) to following:
> {code:sql}
> select hour_start, count(*)
> from realtime_table
> where hour_start >= "2019-09-01 04:00:00" and hour_start < "2019-09-01
> 06:00:00"
> group by hour_start
> {code}
> And he will get result like :
> ||hour_start ||count||
> |2019-09-01 04:00:00 |139202|
> |2019-09-01 05:00:00 |89398|
> And he must convert to a more meaningful result in his mind, it is realy
> annoying!
> ||hour_start ||count||
> |2019-09-01 12:00:00 |139202|
> |2019-09-01 13:00:00 |89398|
> h2. Desgin
> We should not change the way receiver index event, event time should be
> stored in UTC timestamp. We should auto rewrite sql's event time related
> filter.
> In kylin, filter condition in where clause will be convert to a
> *TupleFilter*, and it looks like *RelNode* in Apache Calicate.
> For where hour_start >= "2019-09-01 12:00:00" and hour_start < "2019-09-01
> 14:00:00", we will send TupleFilter to streaming receiver or region server
> which looks like this:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 12:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 14:00:00"
> timestamp
> {noformat}
> But for streaming query, we want to change each ConstantTupleFilter and minus
> value for that timestamp. So the TupleFilter which be sent will be following:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 04:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 06:00:00"
> timestamp
> {noformat}
> Before query result processed by *OLAPEnumerator*, kylin will plus each
> value of time derived column, thus protect row from be filtered by calcite
> generated code.
> So, user will get what he want in his timezone without any burden.
> h2. How to use
> To enable auto shift by time zone, please set
> {color:#DE350B}kylin.stream.auto.just.by.timezone{color} to true.
> You can specific time zone by {color:#DE350B}kylin.web.timezone{color},
> otherwise, time zone will be auto detected.
> Only *time derived column* will be affected.
> h2. Related Issue
> Originally, the event time can only in the format of a long value (UTC
> timestamp). But in some case, the event time is in a format of "-MM-dd
> HH:mm:ss", we use a new class DateTimeParser(introduced in KYLIN-4001) to
> convert such format into a UTC
[jira] [Reopened] (KYLIN-4010) Auto adjust offset according to query server's timezone for time derived column
[
https://issues.apache.org/jira/browse/KYLIN-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen reopened KYLIN-4010:
-
> Auto adjust offset according to query server's timezone for time derived
> column
> ---
>
> Key: KYLIN-4010
> URL: https://issues.apache.org/jira/browse/KYLIN-4010
> Project: Kylin
> Issue Type: Improvement
> Components: Others
>Affects Versions: v3.0.0-alpha
>Reporter: zengrui
>Assignee: Xiaoxiang Yu
>Priority: Minor
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-07-15-17-15-31-209.png,
> image-2019-07-15-17-17-04-029.png, image-2019-07-15-17-17-39-568.png,
> image-2019-09-22-16-35-23-663.png, image-2019-09-22-16-43-19-248.png,
> image-2019-09-22-20-28-02-402.png, image-2019-09-22-20-35-19-038.png,
> image-2019-09-22-20-53-34-720.png, image-2019-09-24-17-23-42-451.png,
> image-2019-09-24-17-33-29-824.png, image-2019-09-24-17-35-47-589.png,
> image-2019-09-24-17-36-39-611.png
>
>
> h2. Backgroud
> In realtime OLAP, we index real-time event in streaming receiver. We know
> that each event must contains a timestamp column (we often call it event
> time), that value should represent when this event was produced. Because
> event maybe come from different timezone and use local timezone is always
> *error-prone*, so we recommend to use a {color:#DE350B}GMT+0{color}
> timestamp(System.currentTimeMillis()) to avoid such issue.
> I think this is good by design, it is easy to understand and always correct.
> But the *side effect* is that, the end user(business manager behind a BI
> tools) are unhappy because he have to use GMT+0 with date/time related filter
> in SQL and should understand the result should be *shifted* with his local
> timezone. It is not user-firendly and inconvenient for normal user. Because
> user may compare query result from different data source and compare them and
> summarize, use GMT+0 may trouble them.
> h2. Example
> For example, kylin user work in *GMT+8* (maybe in Shanghai) want to know some
> metrics which occured from {color:#DE350B}2019-09-01 12:00:00{color} to
> {color:#DE350B}2019-09-01 14:00:00{color} in his {color:#DE350B}local
> timezone{color}, so he has to {color:#DE350B}rewrite{color} his query (with
> eight hour offset) to following:
> {code:sql}
> select hour_start, count(*)
> from realtime_table
> where hour_start >= "2019-09-01 04:00:00" and hour_start < "2019-09-01
> 06:00:00"
> group by hour_start
> {code}
> And he will get result like :
> ||hour_start ||count||
> |2019-09-01 04:00:00 |139202|
> |2019-09-01 05:00:00 |89398|
> And he must convert to a more meaningful result in his mind, it is realy
> annoying!
> ||hour_start ||count||
> |2019-09-01 12:00:00 |139202|
> |2019-09-01 13:00:00 |89398|
> h2. Desgin
> We should not change the way receiver index event, event time should be
> stored in UTC timestamp. We should auto rewrite sql's event time related
> filter.
> In kylin, filter condition in where clause will be convert to a
> *TupleFilter*, and it looks like *RelNode* in Apache Calicate.
> For where hour_start >= "2019-09-01 12:00:00" and hour_start < "2019-09-01
> 14:00:00", we will send TupleFilter to streaming receiver or region server
> which looks like this:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 12:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 14:00:00"
> timestamp
> {noformat}
> But for streaming query, we want to change each ConstantTupleFilter and minus
> value for that timestamp. So the TupleFilter which be sent will be following:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 04:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 06:00:00"
> timestamp
> {noformat}
> Before query result processed by *OLAPEnumerator*, kylin will plus each
> value of time derived column, thus protect row from be filtered by calcite
> generated code.
> So, user will get what he want in his timezone without any burden.
> h2. How to use
> To enable auto shift by time zone, please set
> {color:#DE350B}kylin.stream.auto.just.by.timezone{color} to true.
> You can specific time zone by {color:#DE350B}kylin.web.timezone{color},
> otherwise, time zone will be auto detected.
> Only *time derived column* will be affected.
> h2. Related Issue
> Originally, the event time can only in the format of a long value (UTC
> timestamp). But in some case, the event time is in a format of "-MM-dd
> HH:mm:ss", we use a new class DateTimeParser(introduced in KYLIN-4001) to
> convert such format into a UTC timestamp.
> h3. Old Describe
> In Real-Time Streaming Cube when I
[jira] [Resolved] (KYLIN-1716) leave executing query page action stop bug
[ https://issues.apache.org/jira/browse/KYLIN-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-1716. - Resolution: Fixed > leave executing query page action stop bug > -- > > Key: KYLIN-1716 > URL: https://issues.apache.org/jira/browse/KYLIN-1716 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: Jason Zhong >Assignee: Jason Zhong >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > > at 'Insight' page, when executing query, if you click to other page like > 'Model', will prompt 'You've executing query in current page, are you sure to > leave this page?' ,if you click cancel, you still leave query page > successfully. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4156:
Fix Version/s: (was: v3.0.0)
Future
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: Future
>
> Attachments: image-2019-09-20-10-23-50-928.png,
> image-2019-09-20-10-24-16-071.png, image-2019-09-20-10-24-59-221.png,
> image-2019-09-20-10-25-40-510.png, image-2019-09-20-10-26-51-715.png,
> image-2019-09-20-10-27-09-300.png
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4147) User has project's admin permission but doesn't have permission to see the Storage/Planner/streaming tab in Model page
[ https://issues.apache.org/jira/browse/KYLIN-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4147. - Resolution: Fixed > User has project's admin permission but doesn't have permission to see the > Storage/Planner/streaming tab in Model page > --- > > Key: KYLIN-4147 > URL: https://issues.apache.org/jira/browse/KYLIN-4147 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming, Security >Affects Versions: Future, v3.0.0-alpha >Reporter: wangxiaojing >Assignee: wangxiaojing >Priority: Major > Fix For: v3.0.0 > > > User has project's admin permission but doesn't have permission to see > the Storage/Planner/streaming tab in Model page , This is very unfriendly, > which may lead to the fact that users can't really see the build details of > cube, > especially realtime building , except super admin or integrate Other Third > Party like ranger . > Maybe,it will be more appropriate that user has project's admin permission > can also see the Storage/Planner/streaming tab in Model page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3392: Fix Version/s: (was: v3.0.0) Future > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: Future > > Attachments: KYLIN-3392-2.png, KYLIN-3392.png, kylin-3.0.0-alpha2.png > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4181) Schedule Kylin using Kubernetes
[ https://issues.apache.org/jira/browse/KYLIN-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4181: Fix Version/s: (was: v3.0.0) Future > Schedule Kylin using Kubernetes > --- > > Key: KYLIN-4181 > URL: https://issues.apache.org/jira/browse/KYLIN-4181 > Project: Kylin > Issue Type: New Feature > Components: Others >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > Fix For: Future > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4196) RuntimeException:HBase table default:KYLIN_XXXXX exists
[
https://issues.apache.org/jira/browse/KYLIN-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4196:
Fix Version/s: (was: v3.0.0)
Future
> RuntimeException:HBase table default:KYLIN_X exists
> ---
>
> Key: KYLIN-4196
> URL: https://issues.apache.org/jira/browse/KYLIN-4196
> Project: Kylin
> Issue Type: Improvement
> Components: Storage - HBase
>Affects Versions: all
>Reporter: Xiaoxiang Yu
>Priority: Minor
> Fix For: Future, v2.6.5
>
> Attachments: Jietu20191016-210856.png
>
>
> Some users report that "Create HTable" often failed, and throw exception with
> {code:java}
> org.apache.kylin.engine.mr.exception.HadoopShellException:java.lang.RuntimeException:
> HBase table kylin_prod:KYLIN_F7ZGH9AEXB exists!{code}
>
>
> !Jietu20191016-210856.png!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4053) Compare date with timestamp
[
https://issues.apache.org/jira/browse/KYLIN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4053.
-
Resolution: Fixed
> Compare date with timestamp
> ---
>
> Key: KYLIN-4053
> URL: https://issues.apache.org/jira/browse/KYLIN-4053
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: Future
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0
>
>
> The following SQL will fail.
> {code:sql}
> SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4170) Add cube information and query information when hbase deadline happened
[ https://issues.apache.org/jira/browse/KYLIN-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977226#comment-16977226 ] nichunen commented on KYLIN-4170: - [~灬kestrel灬] Can you please describe your design for the fix of this issue? > Add cube information and query information when hbase deadline happened > --- > > Key: KYLIN-4170 > URL: https://issues.apache.org/jira/browse/KYLIN-4170 > Project: Kylin > Issue Type: Improvement > Components: Storage - HBase >Affects Versions: v2.6.2 >Reporter: 柳攀 >Assignee: 柳攀 >Priority: Major > Fix For: Future > > Original Estimate: 168h > Remaining Estimate: 168h > > When there is a large number of cube query and hbase server is overload, > hbase deadline would happened.But there is no information about cube and > query for us to locate the problem.If we could get cube and query information > in query log, we could solve the query problem quickly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4170) Add cube information and query information when hbase deadline happened
[ https://issues.apache.org/jira/browse/KYLIN-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4170: Fix Version/s: (was: v3.0.0) Future > Add cube information and query information when hbase deadline happened > --- > > Key: KYLIN-4170 > URL: https://issues.apache.org/jira/browse/KYLIN-4170 > Project: Kylin > Issue Type: Improvement > Components: Storage - HBase >Affects Versions: v2.6.2 >Reporter: 柳攀 >Assignee: 柳攀 >Priority: Major > Fix For: Future > > Original Estimate: 168h > Remaining Estimate: 168h > > When there is a large number of cube query and hbase server is overload, > hbase deadline would happened.But there is no information about cube and > query for us to locate the problem.If we could get cube and query information > in query log, we could solve the query problem quickly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-3906) ExecutableManager is spelled as ExecutableManger
[ https://issues.apache.org/jira/browse/KYLIN-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3906. - Resolution: Fixed > ExecutableManager is spelled as ExecutableManger > > > Key: KYLIN-3906 > URL: https://issues.apache.org/jira/browse/KYLIN-3906 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v3.0.0 >Reporter: Yanwen Lin >Priority: Trivial > Labels: patch > Fix For: v3.0.0 > > Attachments: KYLIN-3906.patch > > > As titled, please see attachment for patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4208) RT OLAP kylin.stream.node configure optimization support all receiver can have the same config
[ https://issues.apache.org/jira/browse/KYLIN-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4208. - Resolution: Fixed > RT OLAP kylin.stream.node configure optimization support all receiver can > have the same config > -- > > Key: KYLIN-4208 > URL: https://issues.apache.org/jira/browse/KYLIN-4208 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Reporter: wangxiaojing >Assignee: wangxiaojing >Priority: Major > Fix For: Future, v3.0.0, v3.0.0-beta > > > At present, kylin.stream.node only supports two configuration > format: not config(will use native hostname:defaultPort 7070) or config the > IP: Port. In product env, the port number usually needs to be changed. > If we should change the port ,wo should set like ip:port , it will > lead to different configuration files of different nodes of the entire > receiver cluster, which will cause inconvenience to online operation and > maintenance. > We hope to add a configuration method, such as the disposable port, to > solve the problem that the port can be customized and the configuration of > all receivers can be consistent. At the same time, it is compatible with the > previous configuration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4244) ClassNotFoundException while use org.apache.kylin.engine.mr.common.CubeStatsReader in bash
[
https://issues.apache.org/jira/browse/KYLIN-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4244.
-
Resolution: Fixed
> ClassNotFoundException while use
> org.apache.kylin.engine.mr.common.CubeStatsReader in bash
> --
>
> Key: KYLIN-4244
> URL: https://issues.apache.org/jira/browse/KYLIN-4244
> Project: Kylin
> Issue Type: Bug
>Reporter: ZhouKang
>Assignee: ZhouKang
>Priority: Minor
> Fix For: v3.0.0
>
>
> use org.apache.kylin.engine.mr.common.CubeStatsReader to print estimated size
> for cube
>
> {code:java}
> // code placeholder
> bash ./kylin.sh org.apache.kylin.engine.mr.common.CubeStatsReader {cube_name}
> {code}
> get an Exception
> {code:java}
> // code placeholder
> Exception in thread "main" java.lang.NoClassDefFoundError:
> com/tdunning/math/stats/TDigest
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.current(PercentileSerializer.java:62)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.getStorageBytesEstimate(PercentileSerializer.java:52)
> at
> org.apache.kylin.metadata.datatype.DataType.getStorageBytesEstimate(DataType.java:256)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.estimateCuboidStorageSize(CubeStatsReader.java:251)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMapFromRowCount(CubeStatsReader.java:211)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.getCuboidSizeMap(CubeStatsReader.java:170)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.print(CubeStatsReader.java:273)
> at
> org.apache.kylin.engine.mr.common.CubeStatsReader.main(CubeStatsReader.java:435)
> Caused by: java.lang.ClassNotFoundException: com.tdunning.math.stats.TDigest
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 8 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4246) Wrong results from real-time streaming when an optional field is used as a dimension
[
https://issues.apache.org/jira/browse/KYLIN-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4246.
-
Resolution: Fixed
> Wrong results from real-time streaming when an optional field is used as a
> dimension
>
>
> Key: KYLIN-4246
> URL: https://issues.apache.org/jira/browse/KYLIN-4246
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-alpha
>Reporter: Andras Istvan Nagy
>Priority: Critical
> Fix For: v3.0.0
>
>
> We run into an issue which seems to be related to the real-time streaming
> receiver.
> We have an optional field in the kafka messages, that is, in some cases it
> has a value, in other cases it is missing from the JSON message. This field
> is defined as a dimension and is used in the queries. We were assuming that
> when this field is missing from the JSON message, its value will be
> interpreted as null (as is the case with the Kylin batch engine).
> The results for queries that include this field are correct for those
> segments that have been rebuilt with the Kylin batch engine, but incorrect
> when they are built by the streaming receiver.
> E.g. in the example query below (simplified version of our actual queries),
> optional_field is an optional field, and mandatory_field always has a value.
> In our case, the below query should produce 0 as output, because we have no
> such records where optional_field has a value (is not null), and
> mandatory_field='X'. Still, for those segments that are coming from the
> streaming receiver, we get non-0 values.
> {{select count( * )}}
> {{from movement_events}}
> {{where mandatory_field='X'}}
> {{and optional_field is not null;}}
> Is this a known issue? Can we avoid this somehow, without changing the source
> events?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4246) Wrong results from real-time streaming when an optional field is used as a dimension
[
https://issues.apache.org/jira/browse/KYLIN-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4246:
Fix Version/s: v3.0.0
> Wrong results from real-time streaming when an optional field is used as a
> dimension
>
>
> Key: KYLIN-4246
> URL: https://issues.apache.org/jira/browse/KYLIN-4246
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-alpha
>Reporter: Andras Istvan Nagy
>Priority: Critical
> Fix For: v3.0.0
>
>
> We run into an issue which seems to be related to the real-time streaming
> receiver.
> We have an optional field in the kafka messages, that is, in some cases it
> has a value, in other cases it is missing from the JSON message. This field
> is defined as a dimension and is used in the queries. We were assuming that
> when this field is missing from the JSON message, its value will be
> interpreted as null (as is the case with the Kylin batch engine).
> The results for queries that include this field are correct for those
> segments that have been rebuilt with the Kylin batch engine, but incorrect
> when they are built by the streaming receiver.
> E.g. in the example query below (simplified version of our actual queries),
> optional_field is an optional field, and mandatory_field always has a value.
> In our case, the below query should produce 0 as output, because we have no
> such records where optional_field has a value (is not null), and
> mandatory_field='X'. Still, for those segments that are coming from the
> streaming receiver, we get non-0 values.
> {{select count( * )}}
> {{from movement_events}}
> {{where mandatory_field='X'}}
> {{and optional_field is not null;}}
> Is this a known issue? Can we avoid this somehow, without changing the source
> events?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-4184) Real time OLAP query gets wrong result
[
https://issues.apache.org/jira/browse/KYLIN-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-4184.
---
> Real time OLAP query gets wrong result
> --
>
> Key: KYLIN-4184
> URL: https://issues.apache.org/jira/browse/KYLIN-4184
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-beta
>Reporter: wangxiaojing
>Assignee: wangxiaojing
>Priority: Blocker
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-09-30-13-54-13-312.png
>
>
> * 1、Kylin RT OLAP query error，sometimes result count more than right
> ,sometimes less than right;
> * 2、Kylin RT OLAP query error，some dimissions content error,
> eg:minute_start,hour_start；
> * query sql
> {code:java}
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
> content error result
> !image-2019-09-30-13-54-13-312.png!
> * item error result
> {code:java}
> #this sql can get result with 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START = '2019-09-25 18:32:00' --and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
>
> {code:java}
> #this sql can get result with no items ,it get the error result ,the rigth
> result it's 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4184) Real time OLAP query gets wrong result
[
https://issues.apache.org/jira/browse/KYLIN-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4184.
-
Resolution: Fixed
> Real time OLAP query gets wrong result
> --
>
> Key: KYLIN-4184
> URL: https://issues.apache.org/jira/browse/KYLIN-4184
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-beta
>Reporter: wangxiaojing
>Assignee: wangxiaojing
>Priority: Blocker
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-09-30-13-54-13-312.png
>
>
> * 1、Kylin RT OLAP query error，sometimes result count more than right
> ,sometimes less than right;
> * 2、Kylin RT OLAP query error，some dimissions content error,
> eg:minute_start,hour_start；
> * query sql
> {code:java}
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
> content error result
> !image-2019-09-30-13-54-13-312.png!
> * item error result
> {code:java}
> #this sql can get result with 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START = '2019-09-25 18:32:00' --and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
>
> {code:java}
> #this sql can get result with no items ,it get the error result ,the rigth
> result it's 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4184) Real time OLAP query gets wrong result
[
https://issues.apache.org/jira/browse/KYLIN-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4184:
Fix Version/s: v3.0.0-beta
> Real time OLAP query gets wrong result
> --
>
> Key: KYLIN-4184
> URL: https://issues.apache.org/jira/browse/KYLIN-4184
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-beta
>Reporter: wangxiaojing
>Assignee: wangxiaojing
>Priority: Blocker
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-09-30-13-54-13-312.png
>
>
> * 1、Kylin RT OLAP query error，sometimes result count more than right
> ,sometimes less than right;
> * 2、Kylin RT OLAP query error，some dimissions content error,
> eg:minute_start,hour_start；
> * query sql
> {code:java}
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
> content error result
> !image-2019-09-30-13-54-13-312.png!
> * item error result
> {code:java}
> #this sql can get result with 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START = '2019-09-25 18:32:00' --and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
>
> {code:java}
> #this sql can get result with no items ,it get the error result ,the rigth
> result it's 132 items
> select MINUTE_START, EVENTCODE as NAME, RUNTIME, TASKID, count(1) as CNT from
> TASK_SNAPSHOT
> where MINUTE_START >= '2019-09-25 18:32:00' and MINUTE_START<='2019-09-25
> 18:33:00'
> and EVENTCODE = 'faceAntispam'
> group by MINUTE_START, EVENTCODE, RUNTIME, TASKID order by RUNTIME desc{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-2431) StorageCleanupJob will remove intermediate tables created by other kylin instances
[ https://issues.apache.org/jira/browse/KYLIN-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-2431. - Resolution: Fixed > StorageCleanupJob will remove intermediate tables created by other kylin > instances > -- > > Key: KYLIN-2431 > URL: https://issues.apache.org/jira/browse/KYLIN-2431 > Project: Kylin > Issue Type: Improvement >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > > If QA and PROD instances are in same hive environment, running > StorageCleanupJob on QA will remove intermediate tables created by PROD, > which might fail Kylin jobs of PROD. > A solution is to add metastore name to hive table prefix, then filter table > names with metastore name during cleanup job. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4248) When adding a user, the prompt message is incorrect when the user name is empty.
[ https://issues.apache.org/jira/browse/KYLIN-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4248: Fix Version/s: v3.0.0 > When adding a user, the prompt message is incorrect when the user name is > empty. > > > Key: KYLIN-4248 > URL: https://issues.apache.org/jira/browse/KYLIN-4248 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v3.0.0 >Reporter: wu.kehua >Assignee: wu.kehua >Priority: Minor > Fix For: v3.0.0 > > Attachments: image-2019-11-12-09-57-19-078.png > > > When adding a user, the prompt message is incorrect when the user name is > empty. As follow picture. Then I find out the source code of two pages has > mistake tip information. > !image-2019-11-12-09-57-19-078.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4248) When adding a user, the prompt message is incorrect when the user name is empty.
[ https://issues.apache.org/jira/browse/KYLIN-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4248. - Resolution: Fixed > When adding a user, the prompt message is incorrect when the user name is > empty. > > > Key: KYLIN-4248 > URL: https://issues.apache.org/jira/browse/KYLIN-4248 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v3.0.0 >Reporter: wu.kehua >Assignee: wu.kehua >Priority: Minor > Fix For: v3.0.0 > > Attachments: image-2019-11-12-09-57-19-078.png > > > When adding a user, the prompt message is incorrect when the user name is > empty. As follow picture. Then I find out the source code of two pages has > mistake tip information. > !image-2019-11-12-09-57-19-078.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4097) Throw exception when too many dict slice eviction in AppendTrieDictionary
[ https://issues.apache.org/jira/browse/KYLIN-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4097. - Resolution: Fixed > Throw exception when too many dict slice eviction in AppendTrieDictionary > - > > Key: KYLIN-4097 > URL: https://issues.apache.org/jira/browse/KYLIN-4097 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Minor > Fix For: v3.0.0 > > > When the global dict is too large than the spark executor memory, there will > be too many dict slice evictions and loads in AppendTrieDictionary, and the > build job will be very slow. > It's better to throw an exception in advance in this case. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4242) Usage instructions in 'PasswordPlaceholderConfigurer' doesn't work
[
https://issues.apache.org/jira/browse/KYLIN-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4242.
-
Resolution: Fixed
> Usage instructions in 'PasswordPlaceholderConfigurer' doesn't work
> --
>
> Key: KYLIN-4242
> URL: https://issues.apache.org/jira/browse/KYLIN-4242
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.6.0
>Reporter: xiang zhang
>Priority: Trivial
> Fix For: v3.0.0
>
>
> Usage instructions in 'PasswordPlaceholderConfigurer' tells users to use
> 'java' command to run
> ??org.apache.kylin.rest.security.PasswordPlaceholderConfigurer ??which
> doesn't work at all, it would be more objective to use
> '${KYLIN_HOME}/bin/kylin.sh'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4242) Usage instructions in 'PasswordPlaceholderConfigurer' doesn't work
[
https://issues.apache.org/jira/browse/KYLIN-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4242:
Fix Version/s: (was: Future)
v3.0.0
> Usage instructions in 'PasswordPlaceholderConfigurer' doesn't work
> --
>
> Key: KYLIN-4242
> URL: https://issues.apache.org/jira/browse/KYLIN-4242
> Project: Kylin
> Issue Type: Bug
> Components: Security
>Affects Versions: v2.6.0
>Reporter: xiang zhang
>Priority: Trivial
> Fix For: v3.0.0
>
>
> Usage instructions in 'PasswordPlaceholderConfigurer' tells users to use
> 'java' command to run
> ??org.apache.kylin.rest.security.PasswordPlaceholderConfigurer ??which
> doesn't work at all, it would be more objective to use
> '${KYLIN_HOME}/bin/kylin.sh'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3973.
-
Resolution: Fixed
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3973:
Fix Version/s: v3.0.0-beta
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
[
https://issues.apache.org/jira/browse/KYLIN-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3973.
---
> InvalidProtocolBufferException: Protocol message was too large. May be
> malicious.
> --
>
> Key: KYLIN-3973
> URL: https://issues.apache.org/jira/browse/KYLIN-3973
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> For many queries I receive the following exception.
> {noformat}
> 2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
> client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
> com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
> large. May be malicious. Use CodedInputStream.setSizeLimit() to increase
> the size limit.
> at
> com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
> at
> com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
> at
> com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
> at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
> at
> org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
> at
> org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
> at
> org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
> at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4180) Prevent abnormal CPU usage by limiting flat filters length
[
https://issues.apache.org/jira/browse/KYLIN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4180:
Fix Version/s: (was: v3.0.0-beta)
v3.0.0
> Prevent abnormal CPU usage by limiting flat filters length
> --
>
> Key: KYLIN-4180
> URL: https://issues.apache.org/jira/browse/KYLIN-4180
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0, v2.6.5
>
>
> strong textThe following SQL will lead to the list returned by
> org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
> large, and the process to producing the List will use lots of CPU.
> {code:sql}
> select * from temp.test_filter
> where ((a >= 0 and a <= 1)
> or(a >= 1 and a <= 2)
> or(a >= 2 and a <= 3)
> ...
> or(a >= 300 and a <= 301)
> ) and ((b >= 0 and b <= 1)
> or(b >= 1 and b <= 2)
> or(b >= 2 and b <= 3)
> ...
> or(b >= 300 and b <= 301)
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4191) NPE in StreamingTupleConverter when querying count on a dimension field
[
https://issues.apache.org/jira/browse/KYLIN-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4191:
Fix Version/s: v3.0.0
> NPE in StreamingTupleConverter when querying count on a dimension field
> ---
>
> Key: KYLIN-4191
> URL: https://issues.apache.org/jira/browse/KYLIN-4191
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-alpha
>Reporter: Andras Istvan Nagy
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v3.0.0
>
> Attachments: NPE.txt
>
>
> I am using the real-time olap feature with a lambda datasource definition.
> When executing a query that has a count(dim_field) in the select clause,
> where "dim_field" is one of the dimension fields, I get a NPE from
> org.apache.kylin.stream.core.query.StreamingTupleConverter.
> (According to a colleague, a similar query used to work for some time in our
> environment, and then stopped working, but I can't verify this statement.
> Restarting the Kylin processes or disabling/enabling the cube does not "fix"
> the issue.)
> The relevant part of stacktrace seems to be:
> {{Caused by: java.lang.NullPointerException at
> org.apache.kylin.stream.core.query.StreamingTupleConverter.(StreamingTupleConverter.java:86)}}
> Please see attached full stacktrace.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4178) Job scheduler support safe mode
[ https://issues.apache.org/jira/browse/KYLIN-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4178: Fix Version/s: v3.0.0 > Job scheduler support safe mode > > > Key: KYLIN-4178 > URL: https://issues.apache.org/jira/browse/KYLIN-4178 > Project: Kylin > Issue Type: Improvement >Reporter: ZhouKang >Priority: Major > Fix For: v3.0.0 > > > Job scheduler should support safe mode in case of the HBase cluster change. > In xiaomi, we want update the HBase cluster from hbase0.98 to HBase 2.0. The > history data can be migrated previously, but the job has been submitted will > keep running and write data to the old cluster. So we need a method to ensure > that, job create htable in the old cluster will write data to the old > cluster, and the job have not create htable should not be scheduled. > So we need job scheduler safe mode. Open safe mode before changing cluster > config, the running jobs can run continuous, and the new job cannot be > scheduled. > After all running job finished, we can change the cluster config to the new > one, and rest of job can be scheduled again. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-4113) Remove the surplus allCubes field
[
https://issues.apache.org/jira/browse/KYLIN-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-4113.
---
Fix Version/s: v3.0.0-beta
Resolution: Fixed
> Remove the surplus allCubes field
> -
>
> Key: KYLIN-4113
> URL: https://issues.apache.org/jira/browse/KYLIN-4113
> Project: Kylin
> Issue Type: Improvement
> Components: Web , Website
>Affects Versions: v2.6.1
> Environment: computery：macOS Mojave 10.14.5
>Reporter: Smile鑫
>Assignee: Smile鑫
>Priority: Major
> Labels: easyfix
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-07-25-15-26-05-703.png,
> image-2019-07-25-15-42-39-130.png
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> After selecting the project-name, it will go to the backend to get the cube
> under the current project, but there will be a request to get all the cubes,
> even if the backend does not succeed in the permission control, but this is
> superfluous. I checked and found that all the cubes were obtained only
> because I need to judge whether there is a cube with the same name when
> editing or creating the cube. This should not be done in the front end, but
> the information is submitted to the back end when the cube is created. A
> unified judgment is made by the back end. Otherwise, there will be a
> situation where the number of cubes that can be viewed by the current
> logged-in user's privilege is limited, so it is not complete to determine
> whether the cube is renamed at the time of creation. This leads to the fact
> that even if the cube with the same name is created, you can see two cubes
> with the same name when you can view all cube users (admin). . .
> The extra code path to get the cube request:
>
> {code:java}
> webapp/app/js/controllers/cubeSchema.js{code}
>
> In this position ：
> !image-2019-07-25-15-42-39-130.png!
>
> This code should not be written directly in the CubeSchemaCtrl controller,
> otherwise it will trigger the execution of this code when a page references
> the controller, and there is a case of repeated request to initiate the
> request. I don't know why. Perhaps it is because the other properties or
> methods under this controller change to trigger the execution of this code.
>
> I pass the check backend
>
> {code:java}
> server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java{code}
> The file found that there is a ready-made check interface, and I don't know
> why it is not connected. I refer it to the front end for the request and
> delete the other redundant code.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-4083) Fact Distinct Column Step maybe failed or value lost when hashcode of the UHC column value is Integer.MIN_VALUE
[ https://issues.apache.org/jira/browse/KYLIN-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-4083. --- Resolution: Fixed > Fact Distinct Column Step maybe failed or value lost when hashcode of the UHC > column value is Integer.MIN_VALUE > --- > > Key: KYLIN-4083 > URL: https://issues.apache.org/jira/browse/KYLIN-4083 > Project: Kylin > Issue Type: Bug >Reporter: PENG Zhengshuai >Assignee: PENG Zhengshuai >Priority: Major > > In the Fact Distinct Column Step, kylin uses MR to de-dup the values of > columns. > If the column is UHC (ultra high cardinality) column and the value of the > property *kylin.engine.mr.uhc-reducer-count* has been set greater than *1*, > the Mapper task will write the output of UHC column values to different > reducers by *FactDistinctColumnPartitioner* according to the reducer id > The reducer id will be calculated by hash, the implementation in > *FactDistinctColumnsReducerMapping#getReducerIdForCol()*, in this method, > *the reducer id = reducerBeginIndex + Math.abs(value.hashCode()) % > uhcReducerCount* > When the value.hashCode() is Integer.MIN_VALUE, the > Math.abs(value.hashCode()) return also Integer.MIN_VALUE. Thus the reducer id > may return a negative value. This may cause the FactDistinctColumn step > failed, or the UHC column value may be redirected to another reducer which > not belongs to UHC column > For example: > If a UHC column value is "539019926", its hashcode is Integer.MIN_VALUE. > "539019926".hashCode() == Integer.MIN_VALUE == -2147483648. The > Math.abs(-2147483648) returns -2147483648. > so the reducerId = beginIndex + (-2147483648) % uhcReducerCount. > If the beginIndex is 8, uhcReducerCount is 35.The method > *FactDistinctColumnsReducerMapping#getReducerIdForCol()* will return -15. > To Fix it: convert hashCode() value to *long* when calculating Math.abs() to > avoid Integer.MIN_VALUE instead of *int*. > Because the hashCode() method return an int value, Math.abs(longValue) will > never meet the Long.MIN_VALUE, so it's safe > After fix, *FactDistinctColumnsReducerMapping#getReducerIdForCol()* will > return 31 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4016) Garbage Collection on HBase hbase数据存储在s3但是却找的是hdfs
[ https://issues.apache.org/jira/browse/KYLIN-4016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965556#comment-16965556 ] nichunen commented on KYLIN-4016: - Seems duplicate with https://issues.apache.org/jira/browse/KYLIN-3555 > Garbage Collection on HBase hbase数据存储在s3但是却找的是hdfs > --- > > Key: KYLIN-4016 > URL: https://issues.apache.org/jira/browse/KYLIN-4016 > Project: Kylin > Issue Type: Bug > Components: Storage - HBase >Affects Versions: v2.5.0 >Reporter: Tom_yj >Priority: Major > > Garbage Collection on HBase > hbase数据存储在s3上，清理数据时却找的是hdfs，报文件不存在异常。 > 需要清理的文件在s3上真实存在，hdfs上不存在 > > kylin.properties > kylin.env.hdfs-working-dir=s3://XXX-hive/kylin > kylin.storage.hbase.cluster-fs=s3://XXX-hive/hbase > > > > log > java.io.FileNotFoundException: File > /kylin/kylin_metadata/kylin-d3926099-21bb-6893-1055-6d52f2fe17b7/XXX does not > exist. > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:114) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:964) > at > org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:971) > at > org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.dropHdfsPathOnCluster(HDFSPathGarbageCollectionStep.java:95) > at > org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:65) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:69) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:163) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:113) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3961) Optimize TopN measure merge function to reduce TopNCounter errors
[ https://issues.apache.org/jira/browse/KYLIN-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3961. --- Resolution: Feedback Received > Optimize TopN measure merge function to reduce TopNCounter errors > - > > Key: KYLIN-3961 > URL: https://issues.apache.org/jira/browse/KYLIN-3961 > Project: Kylin > Issue Type: Improvement > Components: Measure - TopN >Affects Versions: v2.5.2 > Environment: Huawei FusionInsight >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Major > Labels: easyfix > Original Estimate: 168h > Remaining Estimate: 168h > > Hi Team: > I use "Top-N "measure to query such sql "select sum(AAA) from BBB group by > CCC,DDD", It is much better than a cube without "Top-N". > In my system, kylin cost just 0.2s to query sql with "Top-N" measure cube; If > without "Top-N" measure it may be cost 10s. > But I find that Top-N measure can be optimized to reduce errors. > I use kylin demo to test "TopN". > I build two cube using "KYLIN_SALES". The first cube has three > dimentions:"SELLER_ID","BUYER_ID" and "PART_DT", has one measures: > "SUM(PRICE)" . The second cube has one dimention:"PART_DT", has twon > measures: "SUM(PRICE)" and "TOPN(10)", the "ORDER|SUM by Column" of > "TOPN(10)" is "PRICE", the "Group by Column" of “TOPN(10)” is "SELLER_ID" > and "BUYER_ID",the "Return Type" of "TOPN(10)" is "Top 10". Then I build cube > from "2012-01-01" to "2014-01-01". > I use same sql to query two cube. I find that 2 cubes have a larger error. > The top5 "SUM PRICE" of first cube without "TopN" is "167.7269", "99.9908", > "99.9888","99.9865","99.978". > The top5 "SUM PRICE" of second cube with "TopN" is > "179.27699...","167.6320...","167.3050...","167.2069...","166.7429...". > Does any one meet same problem? > > Best regards. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-4006) kylin reducer throws IllegalArgumentException
[
https://issues.apache.org/jira/browse/KYLIN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-4006.
---
Resolution: Not A Problem
> kylin reducer throws IllegalArgumentException
> -
>
> Key: KYLIN-4006
> URL: https://issues.apache.org/jira/browse/KYLIN-4006
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0.0
> Environment: linux
>Reporter: yxlhaha123
>Priority: Major
> Labels: kylin
>
> build kylin cube，进行到Extract Fact Table Distinct Columns这一步的时候，抛异常
> 2019-05-16 19:55:32,932 INFO [main]
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer: Received value:
> (^h�Uj�7�yȞ�U��+8\` 2019-05-16 19:55:32,945 ERROR [main]
> org.apache.kylin.engine.mr.KylinReducer: java.lang.IllegalArgumentException:
> '(^h�Uj�7�yȞ�U��+8\`' is not a valid date of pattern '-MM-dd
> HH:mm:ss.SSS' at
> org.apache.kylin.common.util.DateFormat.stringToDate(DateFormat.java:89) at
> org.apache.kylin.common.util.DateFormat.stringToMillis(DateFormat.java:107)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doReduce(FactDistinctColumnsReducer.java:189)
> at
> org.apache.kylin.engine.mr.steps.FactDistinctColumnsReducer.doReduce(FactDistinctColumnsReducer.java:60)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:48) at
> org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by:
> java.text.ParseException: Unparseable date: "(^h�Uj�7�yȞ�U��+8\`"
> does not match (\p\{Nd}++)\Q-\E(\p\{Nd}++)\Q-\E(\p\{Nd}++)\Q
> \E(\p\{Nd}++)\Q:\E(\p\{Nd}++)\Q:\E(\p\{Nd}++)\Q.\E(\p\{Nd}++) at
> org.apache.kylin.job.shaded.org.apache.commons.lang3.time.FastDateParser.parse(FastDateParser.java:304)
> at
> org.apache.kylin.job.shaded.org.apache.commons.lang3.time.FastDateFormat.parse(FastDateFormat.java:497)
> at org.apache.kylin.common.util.DateFormat.stringToDate(DateFormat.java:87)
> ... 12 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-3971) InternalErrorException thrown when querying cube with TOP_N measure
[
https://issues.apache.org/jira/browse/KYLIN-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3971.
---
Resolution: Not A Problem
> InternalErrorException thrown when querying cube with TOP_N measure
> ---
>
> Key: KYLIN-3971
> URL: https://issues.apache.org/jira/browse/KYLIN-3971
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
> Attachments: top_n_measure.png
>
>
> I have created a cube with only one measure TOP_N (except mandatory
> `_COUNT_`) and several dimensions.
> !top_n_measure.png!
> When I query it:
> {noformat}
> SELECT
> "DIC_ARTICLE"."SUPPLIER_NAME" AS "SUPPLIER_NAME",
> SUM("KYLIN_TICKETS"."SALE_VALUE") AS "X__alias__0"
> FROM
> "REPORT_SALES"."KYLIN_TICKETS" "KYLIN_TICKETS"
> LEFT JOIN
> "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE" ON
> ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE")
> LEFT JOIN
> "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
> ("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")
> WHERE
> "KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02'
> GROUP BY
> "DIC_ARTICLE"."SUPPLIER_NAME"
> ORDER BY
> "X__alias__0" DESC
> {noformat}
> I get the following exception:
> {noformat}
> 2019-04-23 09:10:46,618 ERROR [http-nio-7070-exec-3]
> controller.BasicController:63 :
> org.apache.kylin.rest.exception.InternalErrorException: RowType=91,
> ColumnRowType=90
> while executing SQL: "SELECT "DIC_ARTICLE"."SUPPLIER_NAME" AS
> "SUPPLIER_NAME", SUM("KYLIN_TICKETS"."SALE_VALUE") AS
> "X__alias__0"FROM "REPORT_SALES"."KYLIN_TICKETS"
> "KYLIN_TICKETS" LEFT JOIN "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE"
> ON ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE") LEFT
> JOIN "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
> ("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")WHERE
>"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02' GROUP BY
> "DIC_ARTICLE"."SUPPLIER_NAME" ORDER BY "X__alias__0" DESC LIMIT 5"
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:441)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:367)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
> at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
>
[jira] [Closed] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
[ https://issues.apache.org/jira/browse/KYLIN-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3972. --- Resolution: Fixed > Kylin chinese website top navbar always open in mobile mode > --- > > Key: KYLIN-3972 > URL: https://issues.apache.org/jira/browse/KYLIN-3972 > Project: Kylin > Issue Type: Bug > Components: Website >Reporter: Roger >Assignee: Roger >Priority: Major > Attachments: WX20190423-154740.png > > > Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-3767) Print the malformed JSON data consumed from Kafka Topic
[
https://issues.apache.org/jira/browse/KYLIN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3767.
-
Resolution: Fixed
> Print the malformed JSON data consumed from Kafka Topic
> ---
>
> Key: KYLIN-3767
> URL: https://issues.apache.org/jira/browse/KYLIN-3767
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v2.2.0, v2.3.0, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3767.master.001.patch
>
>
> Recently, I found that my cube with streaming data built failed, so I checked
> the syslog in the failed MR job.
> But the log contents didn't help, which is as follows:
> {code:java}
> 2019-01-11 15:12:48,774 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537268
> 2019-01-11 15:12:48,776 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537768
> 2019-01-11 15:12:48,778 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
> 2019-01-11 15:12:48,781 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
> 2019-01-11 15:12:48,783 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
> 2019-01-11 15:12:48,787 ERROR [main]
> org.apache.kylin.source.kafka.TimedJsonStreamParser: error
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException:
> Unrecognized character escape 'h' (code 104)
> at [Source: (org.apache.kylin.common.util.ByteBufferBackedInputStream);
> line: 1, column: 207]
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:640)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:3243)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2452)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2407)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:672)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
> at
> org.apache.kylin.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:112)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:87)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:48)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {code}
> Maybe, the malformed json data should be printed in the syslog, which can
> help me to troubleshooting.
> Just like that:
> {code:java}
> ...
> 2019-01-11 15:12:48,778 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
>
[jira] [Closed] (KYLIN-3767) Print the malformed JSON data consumed from Kafka Topic
[
https://issues.apache.org/jira/browse/KYLIN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3767.
---
> Print the malformed JSON data consumed from Kafka Topic
> ---
>
> Key: KYLIN-3767
> URL: https://issues.apache.org/jira/browse/KYLIN-3767
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v2.2.0, v2.3.0, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3767.master.001.patch
>
>
> Recently, I found that my cube with streaming data built failed, so I checked
> the syslog in the failed MR job.
> But the log contents didn't help, which is as follows:
> {code:java}
> 2019-01-11 15:12:48,774 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537268
> 2019-01-11 15:12:48,776 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537768
> 2019-01-11 15:12:48,778 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
> 2019-01-11 15:12:48,781 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
> 2019-01-11 15:12:48,783 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
> 2019-01-11 15:12:48,787 ERROR [main]
> org.apache.kylin.source.kafka.TimedJsonStreamParser: error
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException:
> Unrecognized character escape 'h' (code 104)
> at [Source: (org.apache.kylin.common.util.ByteBufferBackedInputStream);
> line: 1, column: 207]
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:640)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:3243)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2452)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2407)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:672)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
> at
> org.apache.kylin.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:112)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:87)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:48)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {code}
> Maybe, the malformed json data should be printed in the syslog, which can
> help me to troubleshooting.
> Just like that:
> {code:java}
> ...
> 2019-01-11 15:12:48,778 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
>
[jira] [Closed] (KYLIN-3765) 如何对长字符串进行查询
[ https://issues.apache.org/jira/browse/KYLIN-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3765. --- Resolution: Fixed > 如何对长字符串进行查询 > --- > > Key: KYLIN-3765 > URL: https://issues.apache.org/jira/browse/KYLIN-3765 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: 王振强 >Priority: Major > > 模型中有url字段，需要作为查询结果出现，放在度量列中但，没有可以提供查询的方法函数 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3765) 如何对长字符串进行查询
[ https://issues.apache.org/jira/browse/KYLIN-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965385#comment-16965385 ] nichunen commented on KYLIN-3765: - [~wzq] Hi, please send mail to u...@apache.org or d...@apache.org for the use of Kylin. By the way, using English is more encuraged. > 如何对长字符串进行查询 > --- > > Key: KYLIN-3765 > URL: https://issues.apache.org/jira/browse/KYLIN-3765 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: 王振强 >Priority: Major > > 模型中有url字段，需要作为查询结果出现，放在度量列中但，没有可以提供查询的方法函数 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1295) Add new document to describe key concepts like model/cube/segment etc
[ https://issues.apache.org/jira/browse/KYLIN-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1295. --- Resolution: Fixed Available at http://kylin.apache.org/docs/gettingstarted/concepts.html > Add new document to describe key concepts like model/cube/segment etc > - > > Key: KYLIN-1295 > URL: https://issues.apache.org/jira/browse/KYLIN-1295 > Project: Kylin > Issue Type: Improvement > Components: Documentation >Reporter: liyang >Assignee: Shao Feng Shi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3641) Release 2.5.1
[ https://issues.apache.org/jira/browse/KYLIN-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3641. --- Resolution: Fixed > Release 2.5.1 > - > > Key: KYLIN-3641 > URL: https://issues.apache.org/jira/browse/KYLIN-3641 > Project: Kylin > Issue Type: Task >Reporter: Shao Feng Shi >Assignee: Shao Feng Shi >Priority: Major > > Kylin 2.5.0 has released for 1 month, we collected a lot of good feedbacks > and also several bug reportings. Now it is time to release 2.5.1 . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3618) Enable retry connecting to metastore if not available
[ https://issues.apache.org/jira/browse/KYLIN-3618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3618. --- Resolution: Fixed > Enable retry connecting to metastore if not available > - > > Key: KYLIN-3618 > URL: https://issues.apache.org/jira/browse/KYLIN-3618 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v2.4.1 >Reporter: nichunen >Assignee: nichunen >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3545) Kylin couldn't use table in RDBMS as lookup table for Kafka as data source
[ https://issues.apache.org/jira/browse/KYLIN-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3545. --- Resolution: Fixed > Kylin couldn't use table in RDBMS as lookup table for Kafka as data source > -- > > Key: KYLIN-3545 > URL: https://issues.apache.org/jira/browse/KYLIN-3545 > Project: Kylin > Issue Type: Improvement > Components: RDBMS Source >Affects Versions: v2.4.0 >Reporter: wlxie >Priority: Major > > hi，各位老师， > > 在使用kylin的时候遇到一个问题，请教一下。我想使用kylin从kafka获取事实表数据，然后从mysql获取维度表数据，两者进行关联创建cube。但是目前测试了只支持事实表使用jdbc获取数据源，不支持维度表从jdbc获取数据，请问一下是我配置不当还是目前只能支持事实表从jdbc获取数据源，维度表只支持从hive取吗？如果是这样是否有好的方法，还是我必须把维度表加载到hive。 > 谢谢。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3473) Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/kylin]]
[ https://issues.apache.org/jira/browse/KYLIN-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3473. --- Resolution: Not A Problem > Failed to start component > [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/kylin]] > - > > Key: KYLIN-3473 > URL: https://issues.apache.org/jira/browse/KYLIN-3473 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.2.0 > Environment: hdp3.0 > Apache Hive (version 3.1.0.3.0.0.0-1634) > Hive JDBC (version 3.1.0.3.0.0.0-1634) > Beeline version 3.1.0.3.0.0.0-1634 by Apache Hive > Hbase Version 2.0.0.3.0.0.0-1634 >Reporter: Kailun Zhang >Priority: Major > Labels: build > Original Estimate: 168h > Remaining Estimate: 168h > > Jul 27, 2018 5:11:41 PM org.apache.catalina.core.StandardService startInternal > INFO: Starting service Catalina > Jul 27, 2018 5:11:41 PM org.apache.catalina.core.StandardEngine startInternal > INFO: Starting Servlet Engine: Apache Tomcat/7.0.82 > Jul 27, 2018 5:11:41 PM org.apache.catalina.startup.HostConfig deployWAR > INFO: Deploying web application archive > /DATA/kylin/apache-kylin-2.2.0-bin/tomcat/webapps/kylin.war > Jul 27, 2018 5:11:41 PM org.apache.catalina.core.ContainerBase > addChildInternal > SEVERE: ContainerBase.addChild: start: > org.apache.catalina.LifecycleException: Failed to start component > [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/kylin]] > at > org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:162) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1015) > at > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:991) > at > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) > at > org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127) > at > org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2020) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NoSuchMethodError: > org.apache.jasper.xmlparser.ParserUtils: method ()V not found > at org.apache.jasper.runtime.TldScanner.scanTld(TldScanner.java:600) > at org.apache.jasper.runtime.TldScanner.scanJar(TldScanner.java:457) > at org.apache.jasper.runtime.TldScanner.scanJars(TldScanner.java:694) > at org.apache.jasper.runtime.TldScanner.scanTlds(TldScanner.java:350) > at org.apache.jasper.runtime.TldScanner.onStartup(TldScanner.java:239) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5622) > at > org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) > ... 10 more > Jul 27, 2018 5:11:41 PM org.apache.catalina.startup.HostConfig deployWAR > SEVERE: Error deploying web application archive > /DATA/kylin/apache-kylin-2.2.0-bin/tomcat/webapps/kylin.war > java.lang.IllegalStateException: ContainerBase.addChild: start: > org.apache.catalina.LifecycleException: Failed to start component > [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/kylin]] > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1019) > at > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:991) > at > org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) > at > org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127) > at > org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2020) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Jul 27, 2018 5:11:41 PM org.apache.catalina.startup.HostConfig deployWAR > INFO: Deployment of web application archive > /DATA/kylin/apache-kylin-2.2.0-bin/tomcat/webapps/kylin.war has finished in > 467 ms > Jul 27, 2018 5:11:41 PM org.apache.coyote.AbstractProtocol start > INFO: Starting ProtocolHandler ["http-bio-7070"] > Jul 27, 2018 5:11:41 PM org.apache.coyote.AbstractProtocol
[jira] [Closed] (KYLIN-743) Kylin to be OLAP source of SparkSQL
[ https://issues.apache.org/jira/browse/KYLIN-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-743. -- Resolution: Fixed > Kylin to be OLAP source of SparkSQL > --- > > Key: KYLIN-743 > URL: https://issues.apache.org/jira/browse/KYLIN-743 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Luke Han >Assignee: Jiatao Tao >Priority: Major > Fix For: Backlog > > > Kylin's MOLAP Cube is great to serve query which has pattern and request very > frequently from clients. > To be more efficiency, SparkSQL could leverage Kylin as it's OLAP option so > that SparkSQL could make decision route SQL to Kylin's cube or underlying > data. > This feature require more deep design about integration between Kylin and > SparkSQL. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3389) intersect_count error
[
https://issues.apache.org/jira/browse/KYLIN-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3389.
---
Resolution: Cannot Reproduce
> intersect_count error
> -
>
> Key: KYLIN-3389
> URL: https://issues.apache.org/jira/browse/KYLIN-3389
> Project: Kylin
> Issue Type: Bug
> Components: Measure - Count Distinct
>Affects Versions: v2.2.0
> Environment: hive+kylin+hbase
>Reporter: QingLi
>Priority: Major
> Labels: easyfix
> Fix For: Future
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> select
> city_name,
> intersect_count(server_user, dt,array['2018-05-10'] ) aaa
> from
> table
> where dt>='2018-05-10'
> group by city_name
> return: Error while compiling statement: FAILED: SemanticException [Error
> 10025]: Line 3:0 Expression not in GROUP BY key ''2018-05-10''
>
>
>
> here is cube json:
> {
> "uuid": "5fe19005-7b31-4225-b7b3-62fde2c72256",
> "last_modified": 1527489049614,
> "version": "2.2.0.0",
> "name": "CUBE_RETENTION_RD_LQ",
> "owner": "SY_DW_U_RD",
> "descriptor": "CUBE_RETENTION_RD_LQ",
> "cost": 50,
> "status": "READY",
> "segments": [
> {
> "uuid": "75336a8d-a7b6-4465-bb63-fc3ff59b379e",
> "name": "2018030100_2018052800",
> "storage_location_identifier": "KYLIN_JVERXYS4CG",
> "date_range_start": 151986240,
> "date_range_end": 152746560,
> "source_offset_start": 0,
> "source_offset_end": 0,
> "status": "READY",
> "size_kb": 99,
> "input_records": 7534006,
> "input_records_size": 136364517,
> "last_build_time": 1527489049614,
> "last_build_job_id": "729f584e-66aa-46e3-a5b5-837aef69253c",
> "create_time_utc": 1527488801260,
> "cuboid_shard_nums": {},
> "total_shards": 1,
> "blackout_cuboids": [],
> "binary_signature": null,
> "dictionaries": {
> "M_ORDER_DETAIL.CITY_NAME":
> "/dict/SY_MART_KYLIN.M_ORDER_DETAIL/CITY_NAME/e7373276-ee9a-4308-93cf-ba5f3fafcf23.dict",
> "M_ORDER_DETAIL.DT":
> "/dict/SY_MART_KYLIN.M_ORDER_DETAIL/DT/29ec1575-8497-4df2-bb51-dcb9e7dda1a1.dict"
> },
> "snapshots": null,
> "rowkey_stats": [
> [
> "M_ORDER_DETAIL.CITY_NAME",
> 336,
> 2
> ],
> [
> "M_ORDER_DETAIL.DT",
> 57,
> 1
> ]
> ]
> }
> ],
> "create_time_utc": 1527488778311,
> "cuboid_bytes": null,
> "cuboid_bytes_recommend": null,
> "cuboid_last_optimized": 0,
> "size_kb": 99,
> "input_records_count": 7534006,
> "input_records_size": 136364517
> }
>
>
> what's the problem?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-2458) change a build job to DISCARDED status, the value in Duration column increase
[ https://issues.apache.org/jira/browse/KYLIN-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2458. --- Resolution: Fixed > change a build job to DISCARDED status, the value in Duration column increase > -- > > Key: KYLIN-2458 > URL: https://issues.apache.org/jira/browse/KYLIN-2458 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.6.0 >Reporter: 巧克力黑 >Assignee: Jason Zhong >Priority: Major > Attachments: discard.png, duration.png > > > When i discard a job to DISCARDED status, the job can change to a DISCARDED > status ,but the Duration is increasing all the time, and also > I discard the job when the kylin in a hive beeline process, i found the > hive's beeline process in kylin client is not killed in some situations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2840) unseen cube
[ https://issues.apache.org/jira/browse/KYLIN-2840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2840. --- Resolution: Not A Problem > unseen cube > --- > > Key: KYLIN-2840 > URL: https://issues.apache.org/jira/browse/KYLIN-2840 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v2.1.0 >Reporter: cnwangdp >Assignee: Hongbin Ma >Priority: Major > Attachments: kylinBUG.jpg > > > i use API creat cube,got a errro info about cuboid slop over,after,I can't > drop the model,because the unseen cube not yet drop。 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2839) UI Bug: Rowkey setting is inconsistent with dimensions
[ https://issues.apache.org/jira/browse/KYLIN-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2839. --- Resolution: Fixed > UI Bug: Rowkey setting is inconsistent with dimensions > -- > > Key: KYLIN-2839 > URL: https://issues.apache.org/jira/browse/KYLIN-2839 > Project: Kylin > Issue Type: Bug >Reporter: Mu Kong >Priority: Major > > Hi, thanks for all the work you guys have done for the community. > This issue might be duplicated. > When I edit and change the order of a rowkey in "Advanced Setting". And when > I try to save the cube, kylin tells me that the number of rowkeys and the > number of dimensions aren't the same. > Then when I look into the rowkey setting in "Advanced Setting", the last > rowkey that I edited and moved, is copied into two same rowkeys. > To reproduce this issue: > For dimension A, B, C in the cube. > If I, in the Advanced Setting, > change rowkey C from dict to fix_length and move it to the top of the list. > And then, change rowkey B from dict to fix_length and move it to the second > in the list. > Then I save the cube, kylin tells me there are 4 rowkeys and 3 dimensions so > that it can not > create this cube. > Then, I go back to the Advanced Setting, I can see 4 rowkeys in the list, and > there are two rowkey B in this list. > Please let me know if you need more information. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2712) Hybrid No query results, the direct filtration
[
https://issues.apache.org/jira/browse/KYLIN-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-2712.
---
Resolution: Not A Problem
> Hybrid No query results, the direct filtration
> --
>
> Key: KYLIN-2712
> URL: https://issues.apache.org/jira/browse/KYLIN-2712
> Project: Kylin
> Issue Type: Test
> Components: Metadata
>Affects Versions: v2.0.0
> Environment: hadoop:cdh2.6 hbase:1.0.0 kylin:2.0.0
>Reporter: Guoliang
>Assignee: Shao Feng Shi
>Priority: Major
> Labels: test
> Fix For: v2.0.0
>
> Attachments: all_dim_20170615.json, cube0615new_one.json
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> (1)To create a Hybrid 。The joint 2 cube。
> [hadoop@yzdmp006061 hybrid]$ cat my_hybrid.json
> {
> "uuid": "glc86464-3612-gl72-gl1e-3330126a205a",
> "name": "my_hybrid",
> "realizations": [ {
>"type": "CUBE",
>"realization": "cube0615new_one"
> }, {
> "type": "CUBE",
> "realization": "all_dim_20170615"
> }
> ]
> }
> [hadoop@yzdmp006061 hybrid]$
> (2) xview.json
> Has been added
> {
> "name":"my_hybrid",
> "type" : "HYBRID",
> "realization" : "my_hybrid"
> }
> (3)*{color:red}But I query, but only one cube to take effect.Log in,
> according to Exclude cube cube0615new_one because unmatched dimensions,What
> reason is this{color}*
> 2017-07-05 17:39:25,402 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] routing.QueryRouter:68 : The
> realizations remaining: [my_hybrid(HYBRID)] And the final c
> hosen one is the first one
> 2017-07-05 17:39:25,426 DEBUG [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] enumerator.OLAPEnumerator:109 :
> query storage...
> 2017-07-05 17:39:25,426 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] cube.CubeCapabilityChecker:109 :
> Exclude cube cube0615new_one because unmatched dimensio
> ns
> 2017-07-05 17:39:25,427 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] cube.RawQueryLastHacker:42 : No
> group by and aggregation found in this query, will hack
> some result for better look of output...
> 2017-07-05 17:39:25,427 WARN [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] cube.RawQueryLastHacker:73 : SUM is
> not defined for measure column DW.KYLIN_FACT_USER_AD
> SLOT_20170615.GL, output will be meaningless.
> 2017-07-05 17:39:25,429 DEBUG [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] gtrecord.GTCubeStorageQueryBase:253
> : Does not need storage aggregation
> 2017-07-05 17:39:25,429 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] storage.StorageContext:129 : Enable
> limit (storage push down limit) :4
> 2017-07-05 17:39:25,429 DEBUG [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] gtrecord.GTCubeStorageQueryBase:392
> : Aggregate partition results is not beneficial beca
> use no storage aggregation
> 2017-07-05 17:39:25,430 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] gtrecord.GTCubeStorageQueryBase:151
> : Cuboid identified: cube=all_dim_20170615, cuboidId
> =67108863, groupsD=[DW.KYLIN_FACT_USER_ADSLOT_20170615.IS_NATIVE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.DOMAIN,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.IS_RD, DW.KYLIN_FACT_US
> ER_ADSLOT_20170615.ADSLOT_TYPE, DW.KYLIN_FACT_USER_ADSLOT_20170615.DAY_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.OS_VERSION,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.HOUR_ID, D
> W.KYLIN_FACT_USER_ADSLOT_20170615.ADSLOT_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.ADSLOT_SIZE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.CHANNEL_ID, DW.KYLIN_FACT_USER_ADSLOT_2
> 0170615.WIRELESS_NETWORK_TYPE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.DEVICE_TYPE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.CAMPAIGN_DATE_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.
> MODEL, DW.KYLIN_FACT_USER_ADSLOT_20170615.DOMAIN_CATEGORY_IDS,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.ADSLOT_POSITION,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.BROWER, DW.KYLIN_
> FACT_USER_ADSLOT_20170615.TRADE_TYPE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.CAMPAIGN_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.DOMAIN_NAME,
> DW.KYLIN_FACT_USER_ADSLOT_2017061
> 5.BRAND, DW.KYLIN_FACT_USER_ADSLOT_20170615.CREATIVE_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.MOBILE_CARRIER_ID,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.PLATFORM, DW.KYLIN_FA
> CT_USER_ADSLOT_20170615.FLOW_TYPE,
> DW.KYLIN_FACT_USER_ADSLOT_20170615.PROJECT_ID],
> filterD=[DW.KYLIN_FACT_USER_ADSLOT_20170615.DAY_ID], limitPushdown=4,
> storageAggr=fa
> lse
> 2017-07-05 17:39:25,430 INFO [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] gtrecord.CubeSegmentScanner:56 :
> Init CubeSegmentScanner for segment 2017061400_2017
> 061500
> 2017-07-05 17:39:25,431 ERROR [Query
> fc2bdc97-90e8-46ae-b5a9-9917bba1e3bd-200] dict.TrieDictionary:152 : Not a
[jira] [Closed] (KYLIN-2355) kylin cube build is failing at #3 Step Name: Extract Fact Table Distinct Columns
[ https://issues.apache.org/jira/browse/KYLIN-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2355. --- Resolution: Cannot Reproduce > kylin cube build is failing at #3 Step Name: Extract Fact Table Distinct > Columns > > > Key: KYLIN-2355 > URL: https://issues.apache.org/jira/browse/KYLIN-2355 > Project: Kylin > Issue Type: Bug >Reporter: prasannaP >Priority: Major > > I am new to Kylin,I create kylin model and cube by following url, > http://kylin.apache.org/ > every cube build is failing at this step only,i am not able to find the cause, > #3 Step Name: Extract Fact Table Distinct Columns > i find the following error in logs as, > 2016-12-29 11:50:45,421 ERROR [IPC Server handler 18 on 46096] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1482297779079_0128_m_00_0 - exited : > java.lang.ArrayIndexOutOfBoundsException: -1 > at > org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.putRowKeyToHLL(FactDistinctHiveColumnsMapper.java:179) > at > org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.map(FactDistinctHiveColumnsMapper.java:155) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) > 2016-12-29 11:50:45,421 INFO [IPC Server handler 18 on 46096] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from > attempt_1482297779079_0128_m_00_0: Error: > java.lang.ArrayIndexOutOfBoundsException: -1 > at > org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.putRowKeyToHLL(FactDistinctHiveColumnsMapper.java:179) > at > org.apache.kylin.engine.mr.steps.FactDistinctHiveColumnsMapper.map(FactDistinctHiveColumnsMapper.java:155) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2618) Still dictionary cannot go over 2GB given ForestDict
[ https://issues.apache.org/jira/browse/KYLIN-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2618. --- Resolution: Fixed > Still dictionary cannot go over 2GB given ForestDict > > > Key: KYLIN-2618 > URL: https://issues.apache.org/jira/browse/KYLIN-2618 > Project: Kylin > Issue Type: Improvement >Reporter: liyang >Priority: Major > Labels: scope > > This is because dictionary persists through ResourceStore API and that goes > through a copying process based on byte array. The byte array is limited to > 2GB at most. > The solution ideally should be storing dictionary/snapshot directly to HDFS. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2591) How to Archive the OLD snapshot of the CUBE
[ https://issues.apache.org/jira/browse/KYLIN-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2591. --- Resolution: Not A Problem > How to Archive the OLD snapshot of the CUBE > --- > > Key: KYLIN-2591 > URL: https://issues.apache.org/jira/browse/KYLIN-2591 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: amitav >Assignee: liyang >Priority: Major > Labels: features > > Hi, > My cube size 130 GB as of now and its going to increase .Could you please > suggest what could be the best QUBE size would be in ideal production > environment.How I need to archive the Cube old snapshot . > Regards > Amitav -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2319) CubeDesc is null
[ https://issues.apache.org/jira/browse/KYLIN-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2319. --- Resolution: Cannot Reproduce > CubeDesc is null > > > Key: KYLIN-2319 > URL: https://issues.apache.org/jira/browse/KYLIN-2319 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Miles Ming >Assignee: Dong Li >Priority: Major > Attachments: kylin customer cube script.txt, kylin issue.JPG, > kylin.log > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2538) extract(MONTH from CURRENT_DATE) cannot be selected as a column in sql
[ https://issues.apache.org/jira/browse/KYLIN-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2538. --- Resolution: Fixed Tested, fixed on 3.0-beta > extract(MONTH from CURRENT_DATE) cannot be selected as a column in sql > -- > > Key: KYLIN-2538 > URL: https://issues.apache.org/jira/browse/KYLIN-2538 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Dong Li >Assignee: liyang >Priority: Major > > From calcite SQL reference: CURRENT_DATE is a const to get the value of > current value. > But in Kylin this SQL will fail with exceptions: > select extract(MONTH from CURRENT_DATE), count(*) from test_kylin_fact > Error message: > Error while executing SQL "select extract(MONTH from CURRENT_DATE), count(*) > from test_kylin_fact LIMIT 5": No TblColRef found in > EXTRACT_DATE(FLAG(MONTH), Reinterpret(CURRENT_DATE)) > Stacktrace: > Caused by: java.lang.IllegalStateException: No TblColRef found in > EXTRACT_DATE(FLAG(MONTH), Reinterpret(CURRENT_DATE)) > at > org.apache.kylin.query.relnode.OLAPProjectRel.buildColumnRowType(OLAPProjectRel.java:132) > at > org.apache.kylin.query.relnode.OLAPProjectRel.implementOLAP(OLAPProjectRel.java:117) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:83) > at > org.apache.kylin.query.relnode.OLAPLimitRel.implementOLAP(OLAPLimitRel.java:75) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:83) > at > org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:85) > at > org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1248) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:306) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:203) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2318) query cache is not enabled as expected
[ https://issues.apache.org/jira/browse/KYLIN-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2318. --- Resolution: Cannot Reproduce > query cache is not enabled as expected > -- > > Key: KYLIN-2318 > URL: https://issues.apache.org/jira/browse/KYLIN-2318 > Project: Kylin > Issue Type: Bug >Reporter: XIE FAN >Assignee: Hongbin Ma >Priority: Major > > Query cache is invalid even two queries are exactly the same.I think may > cause by non-standard modification of SQLRequest in QueryService and I'm > trying to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2519) find-hive-dependency.sh script found wrong hive-exec-path caused class not found error
[ https://issues.apache.org/jira/browse/KYLIN-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2519. --- Resolution: Cannot Reproduce > find-hive-dependency.sh script found wrong hive-exec-path caused class not > found error > -- > > Key: KYLIN-2519 > URL: https://issues.apache.org/jira/browse/KYLIN-2519 > Project: Kylin > Issue Type: Bug >Reporter: weiyongzhuang >Priority: Major > > Logging initialized using configuration in > jar:file:/usr/lib/hive/lib/hive-common-1.1.1.jar!/hive-log4j.properties > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/lib/hive/lib/hive-jdbc-1.1.1-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/lib/hive/lib/hive-jdbc-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > Exception in thread "main" java.lang.NoClassDefFoundError: > org/antlr/runtime/RecognitionException > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:392) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1160) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1039) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:702) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > Caused by: java.lang.ClassNotFoundException: > org.antlr.runtime.RecognitionException > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > ... 19 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1690) always returning 0 or 1 for sum(a)/sum(b) for integer type a and b
[ https://issues.apache.org/jira/browse/KYLIN-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1690. --- Resolution: Fixed Tested pass > always returning 0 or 1 for sum(a)/sum(b) for integer type a and b > -- > > Key: KYLIN-1690 > URL: https://issues.apache.org/jira/browse/KYLIN-1690 > Project: Kylin > Issue Type: Bug >Reporter: Hongbin Ma >Assignee: Hongbin Ma >Priority: Major > Labels: newbie > > I want to get a value which is defined as sum(a)/sum(b), how can I do > this kind of anlysis. > Now I build a cube which have sum(a) and sum(b), when I execute “select > sum(a)/sum(b) from table1 group by c” ,the result is wrong. sum(a)/sum(b) the > result is all 0 and sum(b)/sum(a) result is all 1. > MMENE_NAMESUCC ATTSUCC/ATT > CSMME15BZX 336981 368366 1 > CSMME32BZX 338754 366842 1 > CSMME07BZX 687965 747694 1 > CSMME03BHW 703269 747623 1 > CSMME12BZX 705856 764656 1 > CSMME16BHW 1962293142173 1 >MMENE_NAME SUCC ATT ATT/SUCC > CSMME15BZX 336981 368366 0 > CSMME32BZX 338754 366842 0 > CSMME07BZX 687965 747694 0 > CSMME03BHW 703269 747623 0 > CSMME12BZX 705856 764656 0 > CSMME16BHW 1962293142173 0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2156) support filters like a != 0
[ https://issues.apache.org/jira/browse/KYLIN-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2156. --- Resolution: Fixed Tested on 3.0-beta, it's already supported > support filters like a != 0 > --- > > Key: KYLIN-2156 > URL: https://issues.apache.org/jira/browse/KYLIN-2156 > Project: Kylin > Issue Type: Bug >Reporter: Hongbin Ma >Assignee: Hongbin Ma >Priority: Major > Labels: newbie > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-2274) Dundas ODBC Error: SQLColAttribute unknown attr,
[
https://issues.apache.org/jira/browse/KYLIN-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-2274.
---
Resolution: Fixed
> Dundas ODBC Error: SQLColAttribute unknown attr,
>
>
> Key: KYLIN-2274
> URL: https://issues.apache.org/jira/browse/KYLIN-2274
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC
>Affects Versions: v1.6.0
> Environment: Kylin 1.6.0
> ODBC 1.6 64 bits
> Dundas BI: 3.0.2 Windows7 64 bits
>Reporter: albertoramon
>Assignee: Dong Li
>Priority: Major
> Attachments: Q1ON SQuirrel.png, Q2onLylinUI.png, Q2onSQuirrel.png,
> TheProblem.png, dundas.png, odbc-2016-12-06-19-01-51.log
>
>
> I tested [Dundas Software|http://www.dundas.com/dundas-bi]
> (I used Kylin Sample Cube)
> (I try with SQuirreL because it uses JDBC directly, and perhaps this can
> helpful )
> You can read list of tables :)
> !dundas.png!
> But when you try to load data of these, there is two types of errors:
> *Error 1:* (from ODBC Log)
> {quote}
> [INFO ] [2016-12-06.19:05:30]SELECT * FROM "DEFAULT"."KYLIN_CAL_DT" WHERE 1 =
> 2
> [INFO ] [2016-12-06.19:05:30]Successfully done executing the query
> [FATAL] [2016-12-06.19:05:30]SQLColAttribute unknown attr, ColNum: 1, FldID:
> 1211
> {quote}
> This query works OK on Kylin UI and SQuirriel:
> !Q1ON SQuirrel.png!
> \\
> \\
> \\
> *Error 2:* (from ODBC Log)
> {quote}
> [ERROR][2016-12-06.19:30:27]The REST query request failed, the error message
> is: Error while executing SQL "SELECT T."USER_DEFINED_FIELD1" AS "TE0",
> T."SITE_ID" AS "TE1" FROM "DEFAULT"."KYLIN_CAL_DT" T GROUP BY
> T."USER_DEFINED_FIELD1", T."SITE_ID" ORDER BY T."USER_DEFINED_FIELD1",
> T."SITE_ID"": From line 1, column 106 to line 1, column 126: Column
> 'USER_DEFINED_FIELD1' not found in table 'T'
> {quote}
> This Query fails on Kylin UI:
> !Q2onLylinUI.png!
> _ERROR: Column 'USER_DEFINED_FIELD1' not found in table 'T'_
> This Query fails on SQuirrel:
> !Q2onSQuirrel.png!
> _ERROR: ava.sql.SQLException: java.io.IOException: POST failed, error code
> 500 and response:
> {"url":"http://172.17.0.2:7070/kylin/api/query","exception":"Error while
> executing SQL \"SELECT T.\"USER_DEFINED_FIELD1\" AS \"TE0\" , T.\"SITE_ID\"
> AS \"TE1\" FROM \"DEFAULT\".\"KYLIN_CAL_DT\" T GROUP BY
> T.\"USER_DEFINED_FIELD1\", T.\"SITE_ID\" ORDER BY
> T.\"USER_DEFINED_FIELD1\", T.\"SITE_ID\"\": From line 4, column 13 to line 4,
> column 33: Column 'USER_DEFINED_FIELD1' not found in table 'T'"}_
> *The problem on Q2 is:*
> ... was very simple, somebody / Dundas?? is crossing columns with their
> tables tables
> !TheProblem.png!
> these columns are of other table :) why? who?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-2444) Kylin should release different hadoop versions to maven repository
[ https://issues.apache.org/jira/browse/KYLIN-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-2444. --- Resolution: Fixed > Kylin should release different hadoop versions to maven repository > -- > > Key: KYLIN-2444 > URL: https://issues.apache.org/jira/browse/KYLIN-2444 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shao Feng Shi >Priority: Major > > Till 1.6.0, Kylin only release/install one compiled version to maven > repository, they are all compiled with hbase 0.98; For example, > kylin-storage-hbase:1.6.0, actually it is kylin-storage-hbase-0.98:1.6.0. > Expectation: kylin release multiple versions: > kylin-storage-hbase-0.98:, kylin-storage-hbase-1.1:, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
