[jira] [Commented] (KYLIN-4189) Prevent abnormal CPU usage by limiting lookup table's rows
[
https://issues.apache.org/jira/browse/KYLIN-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047665#comment-17047665
]
Temple Zhou commented on KYLIN-4189:
The "group by" clause may cause hard computing to implement "count dinstinct".
If there are too many rows, CPU utility will skyrocket. :(
> Prevent abnormal CPU usage by limiting lookup table's rows
> --
>
> Key: KYLIN-4189
> URL: https://issues.apache.org/jira/browse/KYLIN-4189
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> Although lookup table size is limited, snapshot table with too many rows will
> break the server.
> Such as:
> {code:sql}
> select
> dim.b
> FROM
> fact
> INNER JOIN dim ON fact.B = dim.B
> group by
> dim.b
> {code}
> If the number of rows in column dim.B is million, the query will execute for
> dozens seconds and the CPU utilization is extremely high.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
[
https://issues.apache.org/jira/browse/KYLIN-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4295:
---
Description:
As shown in attachments.
BTW: It is a backend issue, as the results return by restful API are
inconsistent.
{code:json}
{"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
service discovery's state"}
{code}
{code:json}
{"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
service discovery's state"}
{code}
was:As shown in attachments
> Instances displayed on Query Node are inconsistent with Job Node
>
>
> Key: KYLIN-4295
> URL: https://issues.apache.org/jira/browse/KYLIN-4295
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.0.0-alpha2
>Reporter: Temple Zhou
>Priority: Minor
> Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg
>
>
> As shown in attachments.
> BTW: It is a backend issue, as the results return by restful API are
> inconsistent.
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":[],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":[],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
> {code:json}
> {"code":"000","data":{"allNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"jobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"queryNodes":["kylin-query-2.kylin.kylin.svc.cluster.local:7070","kylin-query-1.kylin.kylin.svc.cluster.local:7070","kylin-job-0.kylin.kylin.svc.cluster.local:7070","kylin-query-0.kylin.kylin.svc.cluster.local:7070"],"selectedLeaders":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"],"activeJobNodes":["kylin-job-0.kylin.kylin.svc.cluster.local:7070"]},"msg":"get
> service discovery's state"}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4295) Instances displayed on Query Node are inconsistent with Job Node
Temple Zhou created KYLIN-4295: -- Summary: Instances displayed on Query Node are inconsistent with Job Node Key: KYLIN-4295 URL: https://issues.apache.org/jira/browse/KYLIN-4295 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v3.0.0-alpha2 Reporter: Temple Zhou Attachments: 4691576071454_.pic_hd.jpg, 4731576071552_.pic_hd.jpg As shown in attachments -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4265) SQL tab of cube failed when filter is not empty
Temple Zhou created KYLIN-4265: -- Summary: SQL tab of cube failed when filter is not empty Key: KYLIN-4265 URL: https://issues.apache.org/jira/browse/KYLIN-4265 Project: Kylin Issue Type: Bug Components: Metadata Affects Versions: all Reporter: Temple Zhou Assignee: Temple Zhou Attachments: screenshot.png The SQL tab of Cube with filter condition will be failed to display. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3736) Disable Tableau Probing Queries
[ https://issues.apache.org/jira/browse/KYLIN-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968868#comment-16968868 ] Temple Zhou commented on KYLIN-3736: Hi [~akottisa] , I want to know how you suppress these errors ? Any code have you changed ? BTW: I am trying to reproduce this issue but failed.:( > Disable Tableau Probing Queries > --- > > Key: KYLIN-3736 > URL: https://issues.apache.org/jira/browse/KYLIN-3736 > Project: Kylin > Issue Type: Bug >Reporter: anil kumar >Priority: Major > > Hi Team, > > Is there a configuration property to disable Tableau Probing queries. > Sometimes it is not necessary to probe SQL statements when Tableau publish > the reports. These queries leads confusion and some sort of syntactical > errors / exceptions. > > *sample query :* > CREATE LOCAL TEMPORARY TABLE > "XT01_11B3408A_5074_4E96_B7A9_2D4543569E63_1_Connect" ( "COL" INTEGER ) ON > COMMIT PRESERVE ROWS > DROP TABLE "XT01_11B3408A_5074_4E96_B7A9_2D4543569E63_1_Connect" > SELECT 1 AS "COL" > SELECT TOP 1 "COL" FROM ( SELECT 1 AS "COL" ) AS "CHECKTOP" > SELECT "COL" FROM (SELECT 1 AS "COL") AS "CHECKTOP" LIMIT 1 > SELECT "SUBCOL" AS "COL" FROM ( SELECT 1 AS "SUBCOL" ) "SUBQUERY" GROUP BY 1 > *For example below query returns error like below* > SELECT "SUBCOL" AS "COL" FROM ( SELECT 1 AS "SUBCOL" ) "SUBQUERY" GROUP BY 1 > ERROR [http-bio--exec-7] controller.BasicController:62 > org.apache.kylin.rest.exception.InternalErrorException: group by 1 > > These are *TablueTestQueries* in *QueriesUtil.java* in Utility classes > > Regards, > Anilkumar. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4229) String index out of range -1
[
https://issues.apache.org/jira/browse/KYLIN-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4229:
---
Description:
Build cube with "where clause to filter data from source" will get the error
"String index out of range -1"
{code:java}
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.StringBuffer.setCharAt(StringBuffer.java:255)
at
org.apache.commons.lang3.time.FastDatePrinter$PaddedNumberField.appendTo(FastDatePrinter.java:885)
at
org.apache.commons.lang3.time.FastDatePrinter$PaddedNumberField.appendTo(FastDatePrinter.java:870)
at
org.apache.commons.lang3.time.FastDatePrinter.applyRules(FastDatePrinter.java:493)
at
org.apache.commons.lang3.time.FastDatePrinter.applyRulesToString(FastDatePrinter.java:427)
at
org.apache.commons.lang3.time.FastDatePrinter.format(FastDatePrinter.java:446)
at
org.apache.commons.lang3.time.FastDateFormat.format(FastDateFormat.java:434)
at
org.apache.kylin.common.util.DateFormat.formatToDateStr(DateFormat.java:69)
at
org.apache.kylin.job.JoinedFormatter.setDateEnv(JoinedFormatter.java:74)
at org.apache.kylin.job.JoinedFormatter.(JoinedFormatter.java:56)
at
org.apache.kylin.job.JoinedFlatTable.appendWhereStatement(JoinedFlatTable.java:228)
at
org.apache.kylin.job.JoinedFlatTable.generateSelectDataStatement(JoinedFlatTable.java:164)
at
org.apache.kylin.job.JoinedFlatTable.generateSelectDataStatement(JoinedFlatTable.java:136)
at
org.apache.kylin.job.JoinedFlatTable.generateInsertDataStatement(JoinedFlatTable.java:131)
at
org.apache.kylin.source.hive.HiveInputBase.createFlatHiveTableStep(HiveInputBase.java:319)
at
org.apache.kylin.source.hive.HiveInputBase$BaseBatchCubingInputSide.addStepPhase1_DoCreateFlatTable(HiveInputBase.java:259)
at
org.apache.kylin.source.hive.HiveInputBase$BaseBatchCubingInputSide.addStepPhase1_CreateFlatTable(HiveInputBase.java:89)
at
org.apache.kylin.engine.mr.BatchCubingJobBuilder2.build(BatchCubingJobBuilder2.java:59)
at
org.apache.kylin.engine.mr.MRBatchCubingEngine2.createBatchCubingJob(MRBatchCubingEngine2.java:42)
at
org.apache.kylin.engine.EngineFactory.createBatchCubingJob(EngineFactory.java:61)
at
org.apache.kylin.rest.service.JobService.submitJobInternal(JobService.java:247)
at
org.apache.kylin.rest.service.JobService.submitJob(JobService.java:215)
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:402)
at
org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:362)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:653)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
[jira] [Updated] (KYLIN-4228) Not a valid ID: xxx while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4228:
---
Description:
Executing the SQL with "case when" maybe cause the following error:
Not a valid ID: xxx while executing SQL
null while executing SQL
Can be fixed by setting "kylin.query.enable-dynamic-column" to Ture
{code:java}
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at
org.apache.kylin.rest.service.QueryService.executeRequest(QueryService.java:969)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:672)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:218)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:479)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:439)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:377)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
[jira] [Created] (KYLIN-4229) String index out of range -1
Temple Zhou created KYLIN-4229: -- Summary: String index out of range -1 Key: KYLIN-4229 URL: https://issues.apache.org/jira/browse/KYLIN-4229 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v3.0.0-beta Reporter: Temple Zhou Assignee: Temple Zhou Build cube with "where clause to filter data from source" will get the error "String index out of range -1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4228) Not a valid ID: xxx while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4228:
---
Description:
Executing the SQL with "case when" maybe cause the following error:
Not a valid ID: xxx while executing SQL
null while executing SQL
Can be fixed by setting "kylin.query.enable-dynamic-column" to Ture
{code:log}
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at
org.apache.kylin.rest.service.QueryService.executeRequest(QueryService.java:969)
at
org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:672)
at
org.apache.kylin.rest.service.QueryService.query(QueryService.java:218)
at
org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:479)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:439)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:377)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
[jira] [Created] (KYLIN-4228) Not a valid ID: xxx while executing SQL
Temple Zhou created KYLIN-4228: -- Summary: Not a valid ID: xxx while executing SQL Key: KYLIN-4228 URL: https://issues.apache.org/jira/browse/KYLIN-4228 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v3.0.0-beta Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4202) Tableau Server failed with "Access violation - no RTTI data!"
Temple Zhou created KYLIN-4202: -- Summary: Tableau Server failed with "Access violation - no RTTI data!" Key: KYLIN-4202 URL: https://issues.apache.org/jira/browse/KYLIN-4202 Project: Kylin Issue Type: Bug Affects Versions: v3.0.0-alpha2, v2.6.0, v2.5.0 Reporter: Temple Zhou Assignee: Temple Zhou I can open the reports using Tableau Desktop with *JDBC Driver*, but the reposts will failed if I publish it to Tableau Server. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-1671) Date dimension cannot be formatted in tableau
[ https://issues.apache.org/jira/browse/KYLIN-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951783#comment-16951783 ] Temple Zhou commented on KYLIN-1671: Can be resloved by using JDBC Driver !datevalues.png! > Date dimension cannot be formatted in tableau > - > > Key: KYLIN-1671 > URL: https://issues.apache.org/jira/browse/KYLIN-1671 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Attachments: Screen Shot 2016-05-10 at 2.21.25 PM.png, datevalues.png > > > In this article: > http://vizpainter.com/dates-in-tableau-make-me-want-to-tear-my-hair-out/ > We know that there should be a DATE VALUES section in right-click context > menu of Date Dimension columns. > But in kylin sample data, this section is empty, like attached snapshot. > Currently there're 2 workarounds: > 1. Leverage tableau's multi-level date columns > 2. Set column as discrete, and set the format as: > right click the column -> Default properties -> Date format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-1671) Date dimension cannot be formatted in tableau
[ https://issues.apache.org/jira/browse/KYLIN-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-1671: --- Attachment: datevalues.png > Date dimension cannot be formatted in tableau > - > > Key: KYLIN-1671 > URL: https://issues.apache.org/jira/browse/KYLIN-1671 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Attachments: Screen Shot 2016-05-10 at 2.21.25 PM.png, datevalues.png > > > In this article: > http://vizpainter.com/dates-in-tableau-make-me-want-to-tear-my-hair-out/ > We know that there should be a DATE VALUES section in right-click context > menu of Date Dimension columns. > But in kylin sample data, this section is empty, like attached snapshot. > Currently there're 2 workarounds: > 1. Leverage tableau's multi-level date columns > 2. Set column as discrete, and set the format as: > right click the column -> Default properties -> Date format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4181) Schedule Kylin using Kubernetes
Temple Zhou created KYLIN-4181: -- Summary: Schedule Kylin using Kubernetes Key: KYLIN-4181 URL: https://issues.apache.org/jira/browse/KYLIN-4181 Project: Kylin Issue Type: New Feature Components: Others Reporter: Temple Zhou Assignee: Temple Zhou Fix For: Future -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4180) Prevent abnormal CPU usage by limiting flat filters length
[
https://issues.apache.org/jira/browse/KYLIN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou reopened KYLIN-4180:
The fix solution can be better.
> Prevent abnormal CPU usage by limiting flat filters length
> --
>
> Key: KYLIN-4180
> URL: https://issues.apache.org/jira/browse/KYLIN-4180
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> The following SQL will lead to the list returned by
> org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
> large, and the process to producing the List will use lots of CPU.
> {code:sql}
> select * from temp.test_filter
> where ((a >= 0 and a <= 1)
> or(a >= 1 and a <= 2)
> or(a >= 2 and a <= 3)
> ...
> or(a >= 300 and a <= 301)
> ) and ((b >= 0 and b <= 1)
> or(b >= 1 and b <= 2)
> or(b >= 2 and b <= 3)
> ...
> or(b >= 300 and b <= 301)
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (KYLIN-4180) Prevent abnormal CPU usage by limiting flat filters length
[
https://issues.apache.org/jira/browse/KYLIN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou closed KYLIN-4180.
--
Resolution: Duplicate
> Prevent abnormal CPU usage by limiting flat filters length
> --
>
> Key: KYLIN-4180
> URL: https://issues.apache.org/jira/browse/KYLIN-4180
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> The following SQL will lead to the list returned by
> org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
> large, and the process to producing the List will use lots of CPU.
> {code:sql}
> select * from temp.test_filter
> where ((a >= 0 and a <= 1)
> or(a >= 1 and a <= 2)
> or(a >= 2 and a <= 3)
> ...
> or(a >= 300 and a <= 301)
> ) and ((b >= 0 and b <= 1)
> or(b >= 1 and b <= 2)
> or(b >= 2 and b <= 3)
> ...
> or(b >= 300 and b <= 301)
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4137) Accelerate metadata reloading
[ https://issues.apache.org/jira/browse/KYLIN-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-4137: --- Affects Version/s: (was: v2.6.0) (was: v2.5.0) (was: v2.4.0) (was: v2.3.0) (was: v2.2.0) Description: Now, org.apache.kylin.metadata.cachesync.CachedCrudAssist#reloadAll is using an inappropriate method to deal with MySQLJdbcMetadata. As the every call of reloadAt(path) will access the database, which is almost fine to HBase but really unfriendly to the MySQL(RDBMS). I think we should get all of the resource with single request instead of the separate request to get every resource. was: The logic to generate the table resource path had changed since Kylin 2.2.0, so the table loaded(or reload) after Kylin 2.2.0 will use this format – /table/table_name–project_name.json. However, the resource path with former format( /table/table_name.json) will still exists, which can lead to failed loading during startup progress probably. So, we should cleanup the expired and redundant table resource path. Issue Type: Improvement (was: Bug) Summary: Accelerate metadata reloading (was: Delete expired table resource path in metastore) > Accelerate metadata reloading > - > > Key: KYLIN-4137 > URL: https://issues.apache.org/jira/browse/KYLIN-4137 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: Future >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > > Now, org.apache.kylin.metadata.cachesync.CachedCrudAssist#reloadAll is using > an inappropriate method to deal with MySQLJdbcMetadata. > As the every call of reloadAt(path) will access the database, which is almost > fine to HBase but really unfriendly to the MySQL(RDBMS). > I think we should get all of the resource with single request instead of the > separate request to get every resource. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (KYLIN-4137) Delete expired table resource path in metastore
[ https://issues.apache.org/jira/browse/KYLIN-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-4137: --- Description: The logic to generate the table resource path had changed since Kylin 2.2.0, so the table loaded(or reload) after Kylin 2.2.0 will use this format – /table/table_name–project_name.json. However, the resource path with former format( /table/table_name.json) will still exists, which can lead to failed loading during startup progress probably. So, we should cleanup the expired and redundant table resource path. was: The logic to generate the table resource path had changed since Kylin 2.2.0, so the table loaded(or reload) after Kylin 2.2.0 will use this format – /table/table_name–project_name.json. However, the resource path with former format( /table/table_name.json) will still exists, which can lead to failed loading during startup progress probably. > Delete expired table resource path in metastore > --- > > Key: KYLIN-4137 > URL: https://issues.apache.org/jira/browse/KYLIN-4137 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: Future, v2.2.0, v2.3.0, v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > > The logic to generate the table resource path had changed since Kylin 2.2.0, > so the table loaded(or reload) after Kylin 2.2.0 will use this format – > /table/table_name–project_name.json. > However, the resource path with former format( /table/table_name.json) will > still exists, which can lead to failed loading during startup progress > probably. > So, we should cleanup the expired and redundant table resource path. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (KYLIN-4137) Delete expired table resource path in metastore
[ https://issues.apache.org/jira/browse/KYLIN-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-4137: --- Description: The logic to generate the table resource path had changed since Kylin 2.2.0, so the table loaded(or reload) after Kylin 2.2.0 will use this format – /table/table_name–project_name.json. However, the resource path with former format( /table/table_name.json) will still exists, which can lead to failed loading during startup progress probably. > Delete expired table resource path in metastore > --- > > Key: KYLIN-4137 > URL: https://issues.apache.org/jira/browse/KYLIN-4137 > Project: Kylin > Issue Type: Bug > Components: Metadata >Affects Versions: Future, v2.2.0, v2.3.0, v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > > The logic to generate the table resource path had changed since Kylin 2.2.0, > so the table loaded(or reload) after Kylin 2.2.0 will use this format – > /table/table_name–project_name.json. > However, the resource path with former format( /table/table_name.json) will > still exists, which can lead to failed loading during startup progress > probably. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4137) Delete expired table resource path in metastore
Temple Zhou created KYLIN-4137: -- Summary: Delete expired table resource path in metastore Key: KYLIN-4137 URL: https://issues.apache.org/jira/browse/KYLIN-4137 Project: Kylin Issue Type: Bug Components: Metadata Affects Versions: v2.6.0, v2.5.0, v2.4.0, v2.3.0, v2.2.0, Future Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (KYLIN-4074) Exception in thread "Memcached IO over {MemcachedConnection to ..." java.lang.NullPointerException
[
https://issues.apache.org/jira/browse/KYLIN-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4074:
---
Description:
This exception will lead to the failure of thread "Memcached IO over
\{MemcachedConnection to ...\}",
so, the inputQueue of memcache client will be blocked, and every request will
take 2000ms (timeout).
The log is as follows:
{code:java}
2019-07-01 15:21:24,039 INFO [Query
3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
{code}
was:
This exception will lead to the failure of thread "Memcached IO over
{MemcachedConnection to ...}",
so, the inputQueue of memcache client will be blocked, and every request will
take 2000ms (timeout).
The log is as follows:
{code:java}
2019-07-01 15:21:24,039 INFO [Query
3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
{code}
> Exception in thread "Memcached IO over {MemcachedConnection to ..."
> java.lang.NullPointerException
> --
>
> Key: KYLIN-4074
> URL: https://issues.apache.org/jira/browse/KYLIN-4074
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0, v2.5.0, v2.6.2
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> This exception will lead to the failure of thread "Memcached IO over
> \{MemcachedConnection to ...\}",
> so, the inputQueue of memcache client will be blocked, and every request will
> take 2000ms (timeout).
> The log is as follows:
> {code:java}
> 2019-07-01 15:21:24,039 INFO [Query
> 3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
> Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
> segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-4074) Exception in thread "Memcached IO over {MemcachedConnection to ..." java.lang.NullPointerException
[
https://issues.apache.org/jira/browse/KYLIN-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4074:
---
Description:
This exception will lead to the failure of thread "Memcached IO over
{MemcachedConnection to ...}",
so, the inputQueue of memcache client will be blocked, and every request will
take 2000ms (timeout).
The log is as follows:
{code:java}
2019-07-01 15:21:24,039 INFO [Query
3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
{code}
was:
This exception will lead to the failure of thread "Memcached IO over
{MemcachedConnection to ...}",
so, the inputQueue of memcache client will be blocked, and every request will
take 2000ms (timeout).
The log is as follows:
{code:log}
2019-07-01 15:21:24,039 INFO [Query
3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
{code}
> Exception in thread "Memcached IO over {MemcachedConnection to ..."
> java.lang.NullPointerException
> --
>
> Key: KYLIN-4074
> URL: https://issues.apache.org/jira/browse/KYLIN-4074
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0, v2.5.0, v2.6.2
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> This exception will lead to the failure of thread "Memcached IO over
> {MemcachedConnection to ...}",
> so, the inputQueue of memcache client will be blocked, and every request will
> take 2000ms (timeout).
> The log is as follows:
> {code:java}
> 2019-07-01 15:21:24,039 INFO [Query
> 3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
> Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
> segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4074) Exception in thread "Memcached IO over {MemcachedConnection to ..." java.lang.NullPointerException
Temple Zhou created KYLIN-4074:
--
Summary: Exception in thread "Memcached IO over
{MemcachedConnection to ..." java.lang.NullPointerException
Key: KYLIN-4074
URL: https://issues.apache.org/jira/browse/KYLIN-4074
Project: Kylin
Issue Type: Bug
Affects Versions: v2.6.2, v2.5.0, v2.4.0
Reporter: Temple Zhou
Assignee: Temple Zhou
This exception will lead to the failure of thread "Memcached IO over
{MemcachedConnection to ...}",
so, the inputQueue of memcache client will be blocked, and every request will
take 2000ms (timeout).
The log is as follows:
{code:log}
2019-07-01 15:21:24,039 INFO [Query
3f75c477-9c38-35bf-24a0-656459d9ebb1-212641] v2.CubeHBaseEndpointRPC:197 :
Query-3f75c477-9c38-35bf-24a0-656459d9ebb1: no segment result is cached for
segment:1102_pv_uv_users_di[2016070100_2017010100], take time:2000ms
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-4053) Compare date with timestamp
[
https://issues.apache.org/jira/browse/KYLIN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-4053:
---
Description:
The following SQL will fail.
{code:sql}
SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
{code}
> Compare date with timestamp
> ---
>
> Key: KYLIN-4053
> URL: https://issues.apache.org/jira/browse/KYLIN-4053
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: Future
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> The following SQL will fail.
> {code:sql}
> SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4053) Compare date with timestamp
Temple Zhou created KYLIN-4053: -- Summary: Compare date with timestamp Key: KYLIN-4053 URL: https://issues.apache.org/jira/browse/KYLIN-4053 Project: Kylin Issue Type: Sub-task Affects Versions: Future Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4052) Improve compatibility with the Tableau 2019.x
Temple Zhou created KYLIN-4052: -- Summary: Improve compatibility with the Tableau 2019.x Key: KYLIN-4052 URL: https://issues.apache.org/jira/browse/KYLIN-4052 Project: Kylin Issue Type: Improvement Affects Versions: Future Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4033) Can not access Kerberized Cluster with DebugTomcat
Temple Zhou created KYLIN-4033: -- Summary: Can not access Kerberized Cluster with DebugTomcat Key: KYLIN-4033 URL: https://issues.apache.org/jira/browse/KYLIN-4033 Project: Kylin Issue Type: Improvement Components: Tools, Build and Test Affects Versions: all Reporter: Temple Zhou Assignee: Temple Zhou When I start the Kylin Server using DebugTomcat, the cubing job will fail because of "GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3760) Kylin server self register by zookeeper
[ https://issues.apache.org/jira/browse/KYLIN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou resolved KYLIN-3760. Resolution: Resolved > Kylin server self register by zookeeper > --- > > Key: KYLIN-3760 > URL: https://issues.apache.org/jira/browse/KYLIN-3760 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Temple Zhou >Priority: Major > > Currently, kylin servers in a cluster are predefined and can be updated by > manually setting the "kylin.server.cluster-servers". It's not flexible if the > servers change especially when the servers are able to change automatically > in cloud, like k8s. > It's better for servers to self register themselves in zookeeper. With the > help of zookeeper, kylin servers can watch the event change and know the > whole server layout, which will benefit the *Broadcaster* and future > interesting things, like DAG. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3760) Kylin server self register by zookeeper
[ https://issues.apache.org/jira/browse/KYLIN-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou reassigned KYLIN-3760: -- Assignee: Temple Zhou (was: Zhong Yanghong) > Kylin server self register by zookeeper > --- > > Key: KYLIN-3760 > URL: https://issues.apache.org/jira/browse/KYLIN-3760 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Temple Zhou >Priority: Major > > Currently, kylin servers in a cluster are predefined and can be updated by > manually setting the "kylin.server.cluster-servers". It's not flexible if the > servers change especially when the servers are able to change automatically > in cloud, like k8s. > It's better for servers to self register themselves in zookeeper. With the > help of zookeeper, kylin servers can watch the event change and know the > whole server layout, which will benefit the *Broadcaster* and future > interesting things, like DAG. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-4028) Speed up startup progress using cached dependency
[
https://issues.apache.org/jira/browse/KYLIN-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854349#comment-16854349
]
Temple Zhou commented on KYLIN-4028:
{code:bash}
..
if [[ -z $reload_dependency && `ls -1 ${dir}/cached-* 2>/dev/null | wc -l`
-eq 5 ]]
then
echo "Using cached dependency..."
source ${dir}/cached-hive-dependency.sh
source ${dir}/cached-hbase-dependency.sh
source ${dir}/cached-hadoop-conf-dir.sh
source ${dir}/cached-kafka-dependency.sh
source ${dir}/cached-spark-dependency.sh
else
source ${dir}/find-hive-dependency.sh
source ${dir}/find-hbase-dependency.sh
source ${dir}/find-hadoop-conf-dir.sh
source ${dir}/find-kafka-dependency.sh
source ${dir}/find-spark-dependency.sh
fi
..
{code}
Just set the second parameter to "--reload-dependency", which will reload the
dependency.
> Speed up startup progress using cached dependency
> -
>
> Key: KYLIN-4028
> URL: https://issues.apache.org/jira/browse/KYLIN-4028
> Project: Kylin
> Issue Type: Improvement
> Components: Others
>Affects Versions: all
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Minor
>
> The hive/hadoop/hbase dependencies are not volatile, and finding the
> dependencies every time I start the Kylin server will slow down the startup
> speed.
> So, if there are dependencies generated by previous running, we can use it to
> start the server without finding the dependencies again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-4028) Speed up startup progress using cached dependency
Temple Zhou created KYLIN-4028: -- Summary: Speed up startup progress using cached dependency Key: KYLIN-4028 URL: https://issues.apache.org/jira/browse/KYLIN-4028 Project: Kylin Issue Type: Improvement Components: Others Affects Versions: all Reporter: Temple Zhou Assignee: Temple Zhou The hive/hadoop/hbase dependencies are not volatile, and finding the dependencies every time I start the Kylin server will slow down the startup speed. So, if there are dependencies generated by previous running, we can use it to start the server without finding the dependencies again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3978) InternalErrorException: null with precise count distinct
[
https://issues.apache.org/jira/browse/KYLIN-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839172#comment-16839172
]
Temple Zhou edited comment on KYLIN-3978 at 5/14/19 7:37 AM:
-
This issue is caused by the duplicated *SegmentQueryCacheKey*.
The SegmentQueryCacheKey will be same regardless of the value of
storageContext.isExactAggregation().
However, the storageContext.isExactAggregation() decided the bitmap format
getting from the storage,
so if you have two SQL with different isExactAggregation() and different
bitmap format, the latter will be failed because it will use the same
SegmentCache with incompatible bitmap format.
The related code(BitmapSerializer.java):
{code:java}
@Override
public BitmapCounter deserialize(ByteBuffer in) {
try {
//The length of RoaringBitmap is larger than 12
if (peekLength(in) == RESULT_SIZE) {
int flag = in.getInt();
return factory.newBitmap(in.getLong());
} else {
return factory.newBitmap(in);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
{code}
was (Author: temple.zhou):
This issue is caused by the duplicated *SegmentQueryCacheKey*.
The SegmentQueryCacheKey will be same regardless of the value of
storageContext.isExactAggregation().
However, the storageContext.isExactAggregation() decided the bitmap format
getting from the storage,
so if you have two SQL with different isExactAggregation() and different bitmap
format, the latter will be failed because it will use the same SegmentCache
with Incompatible bitmap format.
> InternalErrorException: null with precise count distinct
>
>
> Key: KYLIN-3978
> URL: https://issues.apache.org/jira/browse/KYLIN-3978
> Project: Kylin
> Issue Type: Bug
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> when kylin.query.segment-cache-enabled=true and sql with precise count
> distinct, the InternalErrorException: null happens sometimes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3978) InternalErrorException: null with precise count distinct
[ https://issues.apache.org/jira/browse/KYLIN-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839172#comment-16839172 ] Temple Zhou commented on KYLIN-3978: This issue is caused by the duplicated *SegmentQueryCacheKey*. The SegmentQueryCacheKey will be same regardless of the value of storageContext.isExactAggregation(). However, the storageContext.isExactAggregation() decided the bitmap format getting from the storage, so if you have two SQL with different isExactAggregation() and different bitmap format, the latter will be failed because it will use the same SegmentCache with Incompatible bitmap format. > InternalErrorException: null with precise count distinct > > > Key: KYLIN-3978 > URL: https://issues.apache.org/jira/browse/KYLIN-3978 > Project: Kylin > Issue Type: Bug >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > > when kylin.query.segment-cache-enabled=true and sql with precise count > distinct, the InternalErrorException: null happens sometimes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
[
https://issues.apache.org/jira/browse/KYLIN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839060#comment-16839060
]
Temple Zhou commented on KYLIN-3887:
Before KYLIN-2982 update_sum_measure_default_return_type,
cubeMeasures.js:
{code:javascript}
switch($scope.newMeasure.function.expression){
case "SUM":
if(colType==="tinyint"||colType==="smallint"||colType==="int"||colType==="bigint"||colType==="integer"){
$scope.newMeasure.function.returntype= 'bigint';
}else{
if(colType.indexOf('decimal')!=-1||colType==="double"||colType==="float"){
$scope.newMeasure.function.returntype= 'decimal(19,4)';
}else{
$scope.newMeasure.function.returntype= 'decimal(14,0)';
}
}
break;
{code}
this method will set the return type to decimal when your column type is double.
So, the temporary solution is to reedit the cube via WEB UI(new version), and
rebuild the cube.
> Query with decimal sum measure of double complied failed after KYLIN-3703
> -
>
> Key: KYLIN-3887
> URL: https://issues.apache.org/jira/browse/KYLIN-3887
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: nichunen
>Priority: Major
> Fix For: Future, v3.0.0
>
> Attachments: image-2019-05-14-11-19-05-514.png
>
>
> After KYLIN-3703, Query with decimal sum measure of double complied failed.
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException:
> Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
[
https://issues.apache.org/jira/browse/KYLIN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3887:
---
Attachment: image-2019-05-14-11-19-05-514.png
> Query with decimal sum measure of double complied failed after KYLIN-3703
> -
>
> Key: KYLIN-3887
> URL: https://issues.apache.org/jira/browse/KYLIN-3887
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: nichunen
>Priority: Major
> Fix For: Future, v3.0.0
>
> Attachments: image-2019-05-14-11-19-05-514.png
>
>
> After KYLIN-3703, Query with decimal sum measure of double complied failed.
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException:
> Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3887) Query with decimal sum measure of double complied failed after KYLIN-3703
[
https://issues.apache.org/jira/browse/KYLIN-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839002#comment-16839002
]
Temple Zhou commented on KYLIN-3887:
[~liushaohui] [~nichunen]
I will review my patch soon.
> Query with decimal sum measure of double complied failed after KYLIN-3703
> -
>
> Key: KYLIN-3887
> URL: https://issues.apache.org/jira/browse/KYLIN-3887
> Project: Kylin
> Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: nichunen
>Priority: Major
> Fix For: Future, v3.0.0
>
>
> After KYLIN-3703, Query with decimal sum measure of double complied failed.
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException:
> Line 112, Column 42: Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3978) InternalErrorException: null with precise count distinct
Temple Zhou created KYLIN-3978: -- Summary: InternalErrorException: null with precise count distinct Key: KYLIN-3978 URL: https://issues.apache.org/jira/browse/KYLIN-3978 Project: Kylin Issue Type: Bug Reporter: Temple Zhou Assignee: Temple Zhou when kylin.query.segment-cache-enabled=true and sql with precise count distinct, the InternalErrorException: null happens sometimes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[
https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814011#comment-16814011
]
Temple Zhou commented on KYLIN-3892:
You can enable this feature by setting the
kylin.job.scheduler.priority-considered=true.
When there are too many jobs running, you can submit the job with
priorityOffset property.
For example:
The number of running jobs beyonds the kylin.job.max-concurrent-jobs(the new
cubing job may be pending), meanwhile you have a cubing job with critical
importance, which is really a dilemma if we can't set the job priority.
But now, all the job will be submitted with priorityOffset 0 (default value)
and you can set the priorityOffset with positive value if the job priority is
higher than normal, on the contrary, set it with negative value.
{code:bash}
curl -X PUT \
http://localhost:7070/kylin/api/cubes/xxx/rebuild \
-H 'Authorization: Basic ***' \
-H 'Content-Type: application/json' \
-H 'cache-control: no-cache' \
-d '{
"buildType": "BUILD",
"startTime": 155269440,
"endTime": 155295360,
"priorityOffset": 10 # The higher the value, the higher the priority
}'
{code}
> Set cubing job priority
> ---
>
> Key: KYLIN-3892
> URL: https://issues.apache.org/jira/browse/KYLIN-3892
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Minor
> Fix For: v2.6.2
>
>
> The cubing job with high priority will be delayed when there are too many
> tasks running.
> So I want to set the job priority for the important cubing jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796732#comment-16796732
]
Temple Zhou edited comment on KYLIN-3474 at 3/21/19 1:49 AM:
-
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
{code:java}
public void update(ManagedUser user) {
try (AutoReadWriteLock.AutoLock l = lock.lockForWrite()) {
ManagedUser exist = userMap.get(user.getUsername());
if (exist != null) {
user.setLastModified(exist.getLastModified());
}
user.setUsername(user.getUsername().toUpperCase(Locale.ROOT));
crud.save(user);
} catch (IOException e) {
throw new RuntimeException("Can not update user.", e);
}
}{code}
The userMap is case-insensitive and Kylin will transform the username to
uppercase(ADMIN) even I authenticate with lowercase (admin)，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
was (Author: temple.zhou):
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username such as ADMIN or admin，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.4.2
>
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do
[jira] [Updated] (KYLIN-3898) Cube level properties are ineffective in the some build steps
[ https://issues.apache.org/jira/browse/KYLIN-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-3898: --- Description: I overwrite the property kylin.source.hive.flat-table-storage-format with "RCFILE" in my cube, but there are no effect, the "#1 Step Name: Create Intermediate Flat Hive Table" still use the configuration in kylin.properties. > Cube level properties are ineffective in the some build steps > - > > Key: KYLIN-3898 > URL: https://issues.apache.org/jira/browse/KYLIN-3898 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > > I overwrite the property kylin.source.hive.flat-table-storage-format with > "RCFILE" in my cube, but there are no effect, the "#1 Step Name: Create > Intermediate Flat Hive Table" still use the configuration in kylin.properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3898) Cube level properties are ineffective in the some build steps
Temple Zhou created KYLIN-3898: -- Summary: Cube level properties are ineffective in the some build steps Key: KYLIN-3898 URL: https://issues.apache.org/jira/browse/KYLIN-3898 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v2.6.0, v2.5.0, v2.4.0 Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796732#comment-16796732
]
Temple Zhou edited comment on KYLIN-3474 at 3/20/19 3:05 AM:
-
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username such as ADMIN or admin，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
was (Author: temple.zhou):
{code:java}
public class KylinUserManager {
private static final Logger logger =
LoggerFactory.getLogger(KylinUserManager.class);
public static KylinUserManager getInstance(KylinConfig config) {
return config.getManager(KylinUserManager.class);
}
// called by reflection
static KylinUserManager newInstance(KylinConfig config) throws IOException {
return new KylinUserManager(config);
}
//
private KylinConfig config;
// user ==> ManagedUser
private CaseInsensitiveStringCache userMap;
private CachedCrudAssist crud;
private AutoReadWriteLock lock = new AutoReadWriteLock();
{code}
The userMap is case-insensitive, so I can authenticate successfully with
uppercase or lowercase username，but
org.apache.kylin.rest.security.ManagedUser#equals compare the username in a
case sensitive method.
org.apache.kylin.rest.security.KylinAuthenticationProvider#authenticate
{code:java}
String username = user.getUsername();
logger.debug("User {} authorities : {}", username,
user.getAuthorities());
if (!userService.userExists(username)) {
userService.createUser(user);
} else if
(!userService.loadUserByUsername(username).equals(user)) {
// in case ldap users changing.
userService.updateUser(user);
}
{code}
Finally, the userService.loadUserByUsername(username).equals(user) will always
be false, and the
org.springframework.security.provisioning.UserDetailsManager#updateUser will be
called many times per second.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.4.2
>
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-3474: --- Affects Version/s: v2.6.1 v2.5.0 > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.1 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.1, v2.4.2 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou reopened KYLIN-3474: The problem still exists. :( > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.1, v2.4.2 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3892) Set cubing job priority
Temple Zhou created KYLIN-3892: -- Summary: Set cubing job priority Key: KYLIN-3892 URL: https://issues.apache.org/jira/browse/KYLIN-3892 Project: Kylin Issue Type: New Feature Components: Job Engine Affects Versions: v2.6.0, v2.5.0, v2.4.0 Reporter: Temple Zhou Assignee: Temple Zhou The cubing job with high priority will be delayed when there are too many tasks running. So I want to set the job priority for the important cubing jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3819) kylin.sh run in the foreground
Temple Zhou created KYLIN-3819: -- Summary: kylin.sh run in the foreground Key: KYLIN-3819 URL: https://issues.apache.org/jira/browse/KYLIN-3819 Project: Kylin Issue Type: Improvement Components: Tools, Build and Test Affects Versions: v2.6.0, v2.5.0, v2.4.0 Reporter: Temple Zhou Assignee: Temple Zhou The "kylin.sh start" can only run kylin in the background, but it should be run in the foreground as well. Because only in this way, both streams can be captured if using a supervision system like daemontools. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3809) Support Zookeeper based rest server discovery
[ https://issues.apache.org/jira/browse/KYLIN-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765725#comment-16765725 ] Temple Zhou commented on KYLIN-3809: Good feature :D +1 > Support Zookeeper based rest server discovery > - > > Key: KYLIN-3809 > URL: https://issues.apache.org/jira/browse/KYLIN-3809 > Project: Kylin > Issue Type: New Feature >Reporter: Liu Shaohui >Priority: Minor > > Currently to broadcast config or meta changes, all kylin servers must be set > in kylin.properties. It's not convenient when adding or removing kylin server > especially in k8s env. > > So we can register the endpoint to zk and make the rest server discovery > automatically. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3703) get negative value when query kylin
[ https://issues.apache.org/jira/browse/KYLIN-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-3703: --- Attachment: KYLIN-3703.master.001.patch > get negative value when query kylin > > > Key: KYLIN-3703 > URL: https://issues.apache.org/jira/browse/KYLIN-3703 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: shengxiaopeng >Assignee: Temple Zhou >Priority: Major > Attachments: KYLIN-3703.master.001.patch > > > version: Kylin 2.5.1 > There is a cube built in real time, one metric is average time consuming, > *the source data does not have a negative value, but the negative value > occurs by hourly aggregation*, and the aggregation is normal by minute > > *1. Metric definition* > |Name|Expression|Parameters|Return Type| > |SUM_FS|SUM|Value:APM_FIRSTSCREEN_WEB.PRF_PL_FS, Type:column|bigint| > > *2.Aggregate sql and results by hour，then negative measure value occurs* > ``` > select > hour_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by hour_start > order by hour_start asc > limit 15000 > ``` > > |hour_start|amount|sum_pl_fs| > |2018-11-23 20:00:00|974466|-1317398890| > *3.Aggregate sql and results by minute* > ``` > select > minute_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by minute_start > order by minute_start asc > limit 15000 > ``` > |hour_start|amount|sum_pl_fs| > |2018-11-23 20:21:00|2629|8107124| > |2018-11-23 20:22:00|12216|36558869| > |2018-11-23 20:23:00|12800|38401450| > |...|略|...| > > 4. I guess this should be caused by crossing the border. > I use the data of each minute to get the hourly data, and the total time is > *2977568406*, which is far from the scope of bigint. > ``` > select > sum(t.amount) amount > ,sum(t.sum_pl_fs) sum_pl_fs > ,sum(t.sum_pl_fs) / sum(t.amount) as avg_pl_fs > from > ( > select > minute_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by minute_start > order by minute_start asc > limit 15000 > ) as t > ``` > |amount|sum_pl_fs|avg_pl_fs| > |974466|2977568406|3055| > > After verification, when cast 2977568406 to int ,then value is -1317398890 > > > *Excuse me, is this question a problem with my configuration or query? How > should it be solved?* > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3703) get negative value when query kylin
[ https://issues.apache.org/jira/browse/KYLIN-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou reassigned KYLIN-3703: -- Assignee: Temple Zhou > get negative value when query kylin > > > Key: KYLIN-3703 > URL: https://issues.apache.org/jira/browse/KYLIN-3703 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.1 >Reporter: shengxiaopeng >Assignee: Temple Zhou >Priority: Major > > version: Kylin 2.5.1 > There is a cube built in real time, one metric is average time consuming, > *the source data does not have a negative value, but the negative value > occurs by hourly aggregation*, and the aggregation is normal by minute > > *1. Metric definition* > |Name|Expression|Parameters|Return Type| > |SUM_FS|SUM|Value:APM_FIRSTSCREEN_WEB.PRF_PL_FS, Type:column|bigint| > > *2.Aggregate sql and results by hour，then negative measure value occurs* > ``` > select > hour_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by hour_start > order by hour_start asc > limit 15000 > ``` > > |hour_start|amount|sum_pl_fs| > |2018-11-23 20:00:00|974466|-1317398890| > *3.Aggregate sql and results by minute* > ``` > select > minute_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by minute_start > order by minute_start asc > limit 15000 > ``` > |hour_start|amount|sum_pl_fs| > |2018-11-23 20:21:00|2629|8107124| > |2018-11-23 20:22:00|12216|36558869| > |2018-11-23 20:23:00|12800|38401450| > |...|略|...| > > 4. I guess this should be caused by crossing the border. > I use the data of each minute to get the hourly data, and the total time is > *2977568406*, which is far from the scope of bigint. > ``` > select > sum(t.amount) amount > ,sum(t.sum_pl_fs) sum_pl_fs > ,sum(t.sum_pl_fs) / sum(t.amount) as avg_pl_fs > from > ( > select > minute_start > ,count(*) as amount > ,sum(prf_pl_fs) as sum_pl_fs > from DY_APM.APM_FIRSTSCREEN_WEB > where 1=1 and minute_start >= '2018-11-23 20:00:00' and minute_start < > '2018-11-23 21:00:00' > group by minute_start > order by minute_start asc > limit 15000 > ) as t > ``` > |amount|sum_pl_fs|avg_pl_fs| > |974466|2977568406|3055| > > After verification, when cast 2977568406 to int ,then value is -1317398890 > > > *Excuse me, is this question a problem with my configuration or query? How > should it be solved?* > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3772) CubeMetaIngester works abnormally
[
https://issues.apache.org/jira/browse/KYLIN-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744709#comment-16744709
]
Temple Zhou commented on KYLIN-3772:
[~Shaofengshi]
Related UT should be improved ;)
> CubeMetaIngester works abnormally
> -
>
> Key: KYLIN-3772
> URL: https://issues.apache.org/jira/browse/KYLIN-3772
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.6.1
>
> Attachments: screenshot-1.png
>
>
> I have used the following cmd to extract the cube named xxx_cube, and the
> result looks good. However...
> {code:bash}
> org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
> /Users/zhoutianpeng/tmp
>
> Dump base package locates at:
> /Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
>
> {code}
> When I was trying to ingest the dump file, I got the following exception:
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: error execute
> org.apache.kylin.tool.CubeMetaIngester. Root cause: null
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at
> org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
> at
> org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 1 more
> {code}
> I found that the model is null in
> org.apache.kylin.cube.model.CubeDesc#getModel
> !screenshot-1.png!
> Then, I found that the reason is that
> org.apache.kylin.metadata.cachesync.CachedCrudAssist#initEntityAfterReload
> throws the java.lang.IllegalStateException: No project found for model
> xxx_model, which contributes to the failure of model init, so the model is
> null.
> Maybe, the root cause is that the extracted files don't contain the
> /project/xxx.json, so "No project found for model xxx_model" and the model is
> null because of the failed init.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3772) CubeMetaIngester works abnormally
[
https://issues.apache.org/jira/browse/KYLIN-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3772:
---
Description:
I have used the following cmd to extract the cube named xxx_cube, and the
result looks good. However...
{code:bash}
org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
/Users/zhoutianpeng/tmp
Dump base package locates at:
/Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
{code}
When I was trying to ingest the dump file, I got the following exception:
{code:java}
Exception in thread "main" java.lang.RuntimeException: error execute
org.apache.kylin.tool.CubeMetaIngester. Root cause: null
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
at
org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 1 more
{code}
I found that the model is null in org.apache.kylin.cube.model.CubeDesc#getModel
!screenshot-1.png!
Then, I found that the reason is that
org.apache.kylin.metadata.cachesync.CachedCrudAssist#initEntityAfterReload
throws the java.lang.IllegalStateException: No project found for model
xxx_model, which contributes to the failure of model init, so the model is null.
Maybe, the root cause is that the extracted files don't contain the
/project/xxx.json, so "No project found for model xxx_model" and the model is
null because of the failed init.
was:
I have used the following cmd to extract the cube named xxx_cube, and the
result looks good. However...
{code:bash}
org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
/Users/zhoutianpeng/tmp
Dump base package locates at:
/Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
{code}
When I was trying to ingest the dump file, I got the following exception:
{code:java}
Exception in thread "main" java.lang.RuntimeException: error execute
org.apache.kylin.tool.CubeMetaIngester. Root cause: null
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
at
org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 1 more
{code}
I found that the model is null
!screenshot-1.png!
> CubeMetaIngester works abnormally
> -
>
> Key: KYLIN-3772
> URL: https://issues.apache.org/jira/browse/KYLIN-3772
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: screenshot-1.png
>
>
> I have used the following cmd to extract the cube named xxx_cube, and the
> result looks good. However...
> {code:bash}
> org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
> /Users/zhoutianpeng/tmp
>
> Dump base package locates at:
> /Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
>
> {code}
> When I was trying to ingest the dump file, I got the following exception:
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: error execute
> org.apache.kylin.tool.CubeMetaIngester. Root cause: null
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at
> org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
> at
> org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 1 more
> {code}
> I found that the model is null in
> org.apache.kylin.cube.model.CubeDesc#getModel
> !screenshot-1.png!
> Then, I found that the reason is that
> org.apache.kylin.metadata.cachesync.CachedCrudAssist#initEntityAfterReload
>
[jira] [Updated] (KYLIN-3772) CubeMetaIngester works abnormally
[
https://issues.apache.org/jira/browse/KYLIN-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3772:
---
Description:
I have used the following cmd to extract the cube named xxx_cube, and the
result looks good. However...
{code:bash}
org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
/Users/zhoutianpeng/tmp
Dump base package locates at:
/Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
{code}
When I was trying to ingest the dump file, I got the following exception:
{code:java}
Exception in thread "main" java.lang.RuntimeException: error execute
org.apache.kylin.tool.CubeMetaIngester. Root cause: null
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
at
org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 1 more
{code}
I found that the model is null
!screenshot-1.png!
> CubeMetaIngester works abnormally
> -
>
> Key: KYLIN-3772
> URL: https://issues.apache.org/jira/browse/KYLIN-3772
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: screenshot-1.png
>
>
> I have used the following cmd to extract the cube named xxx_cube, and the
> result looks good. However...
> {code:bash}
> org.apache.kylin.tool.CubeMetaExtractor -cube xxx_cube -destDir
> /Users/zhoutianpeng/tmp
>
> Dump base package locates at:
> /Users/zhoutianpeng/tmp/base_2019_01_17_11_35_33/base_2019_01_17_11_35_33.zip
>
> {code}
> When I was trying to ingest the dump file, I got the following exception:
> {code:java}
> Exception in thread "main" java.lang.RuntimeException: error execute
> org.apache.kylin.tool.CubeMetaIngester. Root cause: null
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at
> org.apache.kylin.tool.CubeMetaIngester.main(CubeMetaIngester.java:217)
> Caused by: java.lang.NullPointerException
> at
> org.apache.kylin.tool.CubeMetaIngester.injest(CubeMetaIngester.java:151)
> at
> org.apache.kylin.tool.CubeMetaIngester.execute(CubeMetaIngester.java:126)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 1 more
> {code}
> I found that the model is null
> !screenshot-1.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3772) CubeMetaIngester works abnormally
[ https://issues.apache.org/jira/browse/KYLIN-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou updated KYLIN-3772: --- Attachment: screenshot-1.png > CubeMetaIngester works abnormally > - > > Key: KYLIN-3772 > URL: https://issues.apache.org/jira/browse/KYLIN-3772 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3770:
---
Attachment: (was: KYLIN-3770.master.002.patch)
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch, KYLIN-3770.master.003.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3770:
---
Attachment: KYLIN-3770.master.003.patch
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch,
> KYLIN-3770.master.002.patch, KYLIN-3770.master.003.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3772) CubeMetaIngester works abnormally
Temple Zhou created KYLIN-3772: -- Summary: CubeMetaIngester works abnormally Key: KYLIN-3772 URL: https://issues.apache.org/jira/browse/KYLIN-3772 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Affects Versions: v2.6.0, v2.5.0, v2.4.0, v2.3.0 Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3770:
---
Attachment: KYLIN-3770.master.002.patch
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch, KYLIN-3770.master.002.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743650#comment-16743650
]
Temple Zhou commented on KYLIN-3770:
Hi [~Wayne0101] ~
Oh~ It seems better to fix the issue by changing the compression method instead
of the decompression method. I will update the PR right away.:)
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743587#comment-16743587
]
Temple Zhou commented on KYLIN-3770:
Hi [~Wayne0101]
How is the ZIP file generated? I generated the base_2019_01_15_17_45_28.zip
using org.apache.kylin.tool.CubeMetaExtractor.
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3770:
---
Attachment: KYLIN-3770.master.001.patch
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3770.master.001.patch
>
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
[
https://issues.apache.org/jira/browse/KYLIN-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3770:
---
Description:
KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is as
follows:
{code:bash}
unzip base_2019_01_15_17_45_28.zip
Archive: base_2019_01_15_17_45_28.zip
inflating: kylin_env
creating: cube/
inflating: cube/test_global_dict1.json
inflating: info
creating: table/
inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
creating: table_exd/
inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
inflating: commit_SHA1
creating: cube_statistics/
creating: cube_statistics/test_global_dict1/
inflating:
cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
inflating:
cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
creating: model_desc/
inflating: model_desc/test_global_dict1.json
inflating: UUID
creating: cube_desc/
inflating: cube_desc/test_global_dict1.json
{code}
However, before the changing of KYLIN-3401, the unzip dir structure is :
{code:bash}
unzip base_2019_01_15_18_18_26.zip
Archive: base_2019_01_15_18_18_26.zip
inflating: base_2019_01_15_18_18_26/kylin_env
inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
inflating: base_2019_01_15_18_18_26/info
inflating:
base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
inflating:
base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
inflating: base_2019_01_15_18_18_26/commit_SHA1
inflating:
base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
inflating:
base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
inflating: base_2019_01_15_18_18_26/UUID
inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
{code}
This modification led to the CubeMetaIngester will always throw the
IllegalStateException, as the tempFolder.list().length will never be 1.
> ZipFileUtils is not compatible CubeMetaIngester
> ---
>
> Key: KYLIN-3770
> URL: https://issues.apache.org/jira/browse/KYLIN-3770
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.3.0, v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> KYLIN-3401 had changed the zip compress tool, so the unzip dir structure is
> as follows:
> {code:bash}
> unzip base_2019_01_15_17_45_28.zip
> Archive: base_2019_01_15_17_45_28.zip
> inflating: kylin_env
>creating: cube/
> inflating: cube/test_global_dict1.json
> inflating: info
>creating: table/
> inflating: table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
>creating: table_exd/
> inflating: table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: commit_SHA1
>creating: cube_statistics/
>creating: cube_statistics/test_global_dict1/
> inflating:
> cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
> inflating:
> cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>creating: model_desc/
> inflating: model_desc/test_global_dict1.json
> inflating: UUID
>creating: cube_desc/
> inflating: cube_desc/test_global_dict1.json
> {code}
> However, before the changing of KYLIN-3401, the unzip dir structure is :
> {code:bash}
> unzip base_2019_01_15_18_18_26.zip
> Archive: base_2019_01_15_18_18_26.zip
> inflating: base_2019_01_15_18_18_26/kylin_env
> inflating: base_2019_01_15_18_18_26/cube/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/info
> inflating:
> base_2019_01_15_18_18_26/table/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating:
> base_2019_01_15_18_18_26/table_exd/DEFAULT.TEST_GLOBAL_DICT1--tianpeng.json
> inflating: base_2019_01_15_18_18_26/commit_SHA1
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/24e4967f-f635-440f-82e8-d73f7cbb2f2d.seq
>
> inflating:
> base_2019_01_15_18_18_26/cube_statistics/test_global_dict1/adfb447f-4369-4dd8-9d76-94bc7f91d278.seq
>
> inflating: base_2019_01_15_18_18_26/model_desc/test_global_dict1.json
> inflating: base_2019_01_15_18_18_26/UUID
> inflating: base_2019_01_15_18_18_26/cube_desc/test_global_dict1.json
> {code}
> This modification led to the CubeMetaIngester will always throw the
> IllegalStateException, as the tempFolder.list().length will never be 1.
--
This
[jira] [Created] (KYLIN-3770) ZipFileUtils is not compatible CubeMetaIngester
Temple Zhou created KYLIN-3770: -- Summary: ZipFileUtils is not compatible CubeMetaIngester Key: KYLIN-3770 URL: https://issues.apache.org/jira/browse/KYLIN-3770 Project: Kylin Issue Type: Bug Components: Tools, Build and Test Affects Versions: v2.6.0, v2.5.0, v2.4.0, v2.3.0 Reporter: Temple Zhou Assignee: Temple Zhou -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3767) Print the malformed JSON data consumed from Kafka Topic
[
https://issues.apache.org/jira/browse/KYLIN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3767:
---
Attachment: KYLIN-3767.master.001.patch
> Print the malformed JSON data consumed from Kafka Topic
> ---
>
> Key: KYLIN-3767
> URL: https://issues.apache.org/jira/browse/KYLIN-3767
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v2.2.0, v2.3.0, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3767.master.001.patch
>
>
> Print the malformed JSON data consumed from Kafka Topic
> Recently, I found that my cube with streaming data built failed, so I checked
> the syslog in the failed MR job.
> But the log contents didn't help, which is as follows:
> {code:java}
> 2019-01-11 15:12:48,774 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537268
> 2019-01-11 15:12:48,776 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537768
> 2019-01-11 15:12:48,778 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
> 2019-01-11 15:12:48,781 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
> 2019-01-11 15:12:48,783 INFO [main]
> org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
> kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
> 2019-01-11 15:12:48,787 ERROR [main]
> org.apache.kylin.source.kafka.TimedJsonStreamParser: error
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException:
> Unrecognized character escape 'h' (code 104)
> at [Source: (org.apache.kylin.common.util.ByteBufferBackedInputStream);
> line: 1, column: 207]
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:640)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:3243)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2452)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2407)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:672)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
> at
> org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
> at
> org.apache.kylin.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:112)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:87)
> at
> org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:48)
> at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {code}
> Maybe, the malformed json data should be printed in the syslog, which can
> help me to troubleshooting.
> Just like that:
> {code:java}
> ...
> 2019-01-11 15:12:48,778
[jira] [Updated] (KYLIN-3767) Print the malformed JSON data consumed from Kafka Topic
[
https://issues.apache.org/jira/browse/KYLIN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3767:
---
Description:
Recently, I found that my cube with streaming data built failed, so I checked
the syslog in the failed MR job.
But the log contents didn't help, which is as follows:
{code:java}
2019-01-11 15:12:48,774 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537268
2019-01-11 15:12:48,776 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537768
2019-01-11 15:12:48,778 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
2019-01-11 15:12:48,781 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
2019-01-11 15:12:48,783 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
2019-01-11 15:12:48,787 ERROR [main]
org.apache.kylin.source.kafka.TimedJsonStreamParser: error
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException:
Unrecognized character escape 'h' (code 104)
at [Source: (org.apache.kylin.common.util.ByteBufferBackedInputStream); line:
1, column: 207]
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:640)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:3243)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2452)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2407)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:672)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
at
org.apache.kylin.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:112)
at
org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:87)
at
org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:48)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
{code}
Maybe, the malformed json data should be printed in the syslog, which can help
me to troubleshooting.
Just like that:
{code:java}
...
2019-01-11 15:12:48,778 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
2019-01-11 15:12:48,781 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
2019-01-11 15:12:48,783 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
2019-01-11 15:12:48,785 ERROR [main]
org.apache.kylin.source.kafka.TimedJsonStreamParser: malformed data:
{"site":"10010-2","channel":"3","atime":1547119709319,"userid":"909c1c003ee825fc57c9d1fb20f279091547119221751;declare
@q varchar(99);set
[jira] [Created] (KYLIN-3767) Print the malformed JSON data consumed from Kafka Topic
Temple Zhou created KYLIN-3767:
--
Summary: Print the malformed JSON data consumed from Kafka Topic
Key: KYLIN-3767
URL: https://issues.apache.org/jira/browse/KYLIN-3767
Project: Kylin
Issue Type: Improvement
Components: Job Engine
Affects Versions: v2.4.0, v2.3.0, v2.2.0
Reporter: Temple Zhou
Assignee: Temple Zhou
Print the malformed JSON data consumed from Kafka Topic
Recently, I found that my cube with streaming data built failed, so I checked
the syslog in the failed MR job.
But the log contents didn't help, which is as follows:
{code:java}
2019-01-11 15:12:48,774 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537268
2019-01-11 15:12:48,776 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1537768
2019-01-11 15:12:48,778 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
2019-01-11 15:12:48,781 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
2019-01-11 15:12:48,783 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1539268
2019-01-11 15:12:48,787 ERROR [main]
org.apache.kylin.source.kafka.TimedJsonStreamParser: error
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException:
Unrecognized character escape 'h' (code 104)
at [Source: (org.apache.kylin.common.util.ByteBufferBackedInputStream); line:
1, column: 207]
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:640)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeEscaped(UTF8StreamJsonParser.java:3243)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2452)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2407)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:269)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:672)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
at
org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3072)
at
org.apache.kylin.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:112)
at
org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:87)
at
org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper.doMap(KafkaFlatTableMapper.java:48)
at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
{code}
Maybe, the malformed json data should be printed in the syslog, which can help
me to troubleshooting.
Just like that:
{code:java}
...
2019-01-11 15:12:48,778 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538268
2019-01-11 15:12:48,781 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset 1538768
2019-01-11 15:12:48,783 INFO [main]
org.apache.kylin.source.kafka.hadoop.KafkaInputRecordReader:
kylin-full-site-pvuv:kafka4:9092:2 fetching offset
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733998#comment-16733998 ] Temple Zhou commented on KYLIN-3474: Hi [~Aron.tao] Oh, you are right, my kylin version is 2.4.0, and the KYLIN-3562 had fixed the issue, I will close it(y) > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou closed KYLIN-3474. -- Resolution: Duplicate Fix Version/s: v2.4.1 v2.4.2 > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Fix For: v2.4.2, v2.4.1 > > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Description:
I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
Tableau Server will get the malformed token error when do query via restful API.
However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
error will be gone.
was:
I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
Tableau Server will get the malformed token error when do query via restful API.
However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
error will be gone.
Now, I just annotate the following
lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and the
Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
{code:java}
@Override
public Authentication authenticate(Authentication authentication) throws
AuthenticationException {
byte[] hashKey = hf.hashString(authentication.getName() +
authentication.getCredentials()).asBytes();
String userKey = Arrays.toString(hashKey);
// if (userService.isEvictCacheFlag()) {
// userCache.invalidateAll();
// userService.setEvictCacheFlag(false);
// }
Authentication authed = userCache.getIfPresent(userKey);
if (null != authed) {
SecurityContextHolder.getContext().setAuthentication(authed);
} else {
{code}
I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Affects Version/s: v2.3.0
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.3.0, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733915#comment-16733915
]
Temple Zhou commented on KYLIN-3474:
Hello~
Can anyone review the patch?
I remove the following lines
{code:java}
public void updateUser(UserDetails user) {
Preconditions.checkState(user instanceof ManagedUser, "User {} is not
ManagedUser", user);
ManagedUser managedUser = (ManagedUser) user;
getKylinUserManager().update(managedUser);
logger.trace("update user : {}", user.getUsername());
setEvictCacheFlag(true); // removed
}
{code}
As I found that "setEvictCacheFlag(true)" will made the userCache in
KylinAuthenticationProvider invalid forever. Every RESTfull API Call will be
auth by authenticationProvider instead of using the userCache, which will
contribute to the malformed token with large concurrency and let the LDAP
server more pressure.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Attachment: KYLIN-3474.master.001.patch
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3474.master.001.patch, kylin.log,
> tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Attachment: kylin.log
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: kylin.log, tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Attachment: tableau-malformed-token.png
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: tableau-malformed-token.png
>
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3501) Cube Data Exporter & Importer
[ https://issues.apache.org/jira/browse/KYLIN-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Temple Zhou reassigned KYLIN-3501: -- Assignee: Temple Zhou > Cube Data Exporter & Importer > - > > Key: KYLIN-3501 > URL: https://issues.apache.org/jira/browse/KYLIN-3501 > Project: Kylin > Issue Type: Wish > Components: Tools, Build and Test >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > > I think the performance problems is inevitable. So, I often migrate the cube > from PROD ENV to DEV ENV for performance tunning. > My steps: > 1. find the related metadata of the cube and export the metadata; > 2. find the related hive table and export the table data; > 3. import the cube metadata and load the hive table data > 4. build the cube > How about integrating the steps into a tool? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou reassigned KYLIN-3474:
--
Assignee: Temple Zhou
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729341#comment-16729341
]
Temple Zhou commented on KYLIN-3430:
[~kangkaisen] Thanks for you reminder~
I have updated the attachment, please review it again. :D
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.001.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: (was: KYLIN-3430.master.002.patch)
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.001.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: KYLIN-3430.master.001.patch
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.001.patch, KYLIN-3430.master.002.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: (was: KYLIN-3430.master.001.patch)
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.002.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: KYLIN-3430.master.002.patch
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.002.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: KYLIN-3430.master.001.patch
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.001.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3430:
---
Attachment: (was: KYLIN-3430.master.001.patch)
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3718) Segments in volatile range is more than designated value
[
https://issues.apache.org/jira/browse/KYLIN-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3718:
---
Priority: Major (was: Minor)
> Segments in volatile range is more than designated value
>
>
> Key: KYLIN-3718
> URL: https://issues.apache.org/jira/browse/KYLIN-3718
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.3.0, v2.4.0, v2.5.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3718.master.001.patch
>
>
> I have designated the volatile value of the cube to 1 and the auto merge
> thresholds is 3, but I found the former 3 segments was not merged when the
> 4th segment was ready.
> the ready segments are as below:
> # xxx[2018120100_2018120200]
> # xxx[2018120200_2018120300]
> # xxx[2018120300_2018120400]
> # xxx[2018120400_2018120500]
> Besides, I found the log info:
> {code:java}
> segment in volatile range: xxx[2018120300_2018120400]
> segment in volatile range: xxx[2018120400_2018120500]{code}
> I think the expected result should be:
> seg: xxx[2018120400_2018120500] should be reserved
> seg:
> xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
> should be merged.
> However, the truth is:
> When the 5th segments was ready, the former 3 segments were automatically
> merged.
> Although the value of volatile range is 1, there are always 2 segments will
> be reserved.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3718) Segments in volatile range is more than designated value
[
https://issues.apache.org/jira/browse/KYLIN-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3718:
---
Description:
I have designated the volatile value of the cube to 1 and the auto merge
thresholds is 3, but I found the former 3 segments was not merged when the 4th
segment was ready.
the ready segments are as below:
# xxx[2018120100_2018120200]
# xxx[2018120200_2018120300]
# xxx[2018120300_2018120400]
# xxx[2018120400_2018120500]
Besides, I found the log info:
{code:java}
segment in volatile range: xxx[2018120300_2018120400]
segment in volatile range: xxx[2018120400_2018120500]{code}
I think the expected result should be:
seg: xxx[2018120400_2018120500] should be reserved
seg:
xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
should be merged.
However, the truth is:
When the 5th segments was ready, the former 3 segments were automatically
merged.
Although the value of volatile range is 1, there are always 2 segments will be
reserved.
was:
I have designated the volatile value of the cube to 1 and the auto merge
thresholds is 3, but I found the former 3 segments was not merged when the 4th
segment was ready.
the ready segments are as below:
# xxx[2018120100_2018120200]
# xxx[2018120200_2018120300]
# xxx[2018120300_2018120400]
# xxx[2018120400_2018120500]
Besides, I found the log info:
{code:java}
2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
volatile range: xxx[2018120300_2018120400]rangeStart:154396800,
rangeEnd154405440
2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
volatile range: xxx[2018120400_2018120500]rangeStart:154405440,
rangeEnd154414080{code}
I think the expected result should be:
seg: xxx[2018120400_2018120500] should be reserved
seg:
xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
should be merged.
However, the truth is:
When the 5th segments was ready, the former 3 segments were automatically
merged.
Although the value of volatile range is 1, there are always 2 segments will be
reserved.
> Segments in volatile range is more than designated value
>
>
> Key: KYLIN-3718
> URL: https://issues.apache.org/jira/browse/KYLIN-3718
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.3.0, v2.4.0, v2.5.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Minor
> Attachments: KYLIN-3718.master.001.patch
>
>
> I have designated the volatile value of the cube to 1 and the auto merge
> thresholds is 3, but I found the former 3 segments was not merged when the
> 4th segment was ready.
> the ready segments are as below:
> # xxx[2018120100_2018120200]
> # xxx[2018120200_2018120300]
> # xxx[2018120300_2018120400]
> # xxx[2018120400_2018120500]
> Besides, I found the log info:
> {code:java}
> segment in volatile range: xxx[2018120300_2018120400]
> segment in volatile range: xxx[2018120400_2018120500]{code}
> I think the expected result should be:
> seg: xxx[2018120400_2018120500] should be reserved
> seg:
> xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
> should be merged.
> However, the truth is:
> When the 5th segments was ready, the former 3 segments were automatically
> merged.
> Although the value of volatile range is 1, there are always 2 segments will
> be reserved.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3718) Segments in volatile range is more than designated value
[
https://issues.apache.org/jira/browse/KYLIN-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3718:
---
Attachment: KYLIN-3718.master.001.patch
> Segments in volatile range is more than designated value
>
>
> Key: KYLIN-3718
> URL: https://issues.apache.org/jira/browse/KYLIN-3718
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.3.0, v2.4.0, v2.5.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Minor
> Attachments: KYLIN-3718.master.001.patch
>
>
> I have designated the volatile value of the cube to 1 and the auto merge
> thresholds is 3, but I found the former 3 segments was not merged when the
> 4th segment was ready.
> the ready segments are as below:
> # xxx[2018120100_2018120200]
> # xxx[2018120200_2018120300]
> # xxx[2018120300_2018120400]
> # xxx[2018120400_2018120500]
> Besides, I found the log info:
> {code:java}
> 2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
> volatile range: xxx[2018120300_2018120400]rangeStart:154396800,
> rangeEnd154405440
> 2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
> volatile range: xxx[2018120400_2018120500]rangeStart:154405440,
> rangeEnd154414080{code}
> I think the expected result should be:
> seg: xxx[2018120400_2018120500] should be reserved
> seg:
> xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
> should be merged.
> However, the truth is:
> When the 5th segments was ready, the former 3 segments were automatically
> merged.
> Although the value of volatile range is 1, there are always 2 segments will
> be reserved.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3718) Segments in volatile range is more than designated value
Temple Zhou created KYLIN-3718:
--
Summary: Segments in volatile range is more than designated value
Key: KYLIN-3718
URL: https://issues.apache.org/jira/browse/KYLIN-3718
Project: Kylin
Issue Type: Bug
Components: Metadata
Affects Versions: v2.5.0, v2.4.0, v2.3.0
Reporter: Temple Zhou
Assignee: Temple Zhou
I have designated the volatile value of the cube to 1 and the auto merge
thresholds is 3, but I found the former 3 segments was not merged when the 4th
segment was ready.
the ready segments are as below:
# xxx[2018120100_2018120200]
# xxx[2018120200_2018120300]
# xxx[2018120300_2018120400]
# xxx[2018120400_2018120500]
Besides, I found the log info:
{code:java}
2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
volatile range: xxx[2018120300_2018120400]rangeStart:154396800,
rangeEnd154405440
2018-12-13 16:30:48,065 WARN [Thread-799] model.Segments:182 : segment in
volatile range: xxx[2018120400_2018120500]rangeStart:154405440,
rangeEnd154414080{code}
I think the expected result should be:
seg: xxx[2018120400_2018120500] should be reserved
seg:
xxx[2018120100_2018120200],xxx[2018120200_2018120300],xxx[2018120300_2018120400]
should be merged.
However, the truth is:
When the 5th segments was ready, the former 3 segments were automatically
merged.
Although the value of volatile range is 1, there are always 2 segments will be
reserved.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591246#comment-16591246
]
Temple Zhou commented on KYLIN-3460:
emm... Fix Version is 2.4.0 ? R U sure ? I use Kylin 2.4.0 and the issue is
existing.
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.0
>
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16574354#comment-16574354
]
Temple Zhou commented on KYLIN-3474:
[~albertoramon]
I'm sorry, I can't find a solution except annotating the above lines. As even I
conf the session persistence in the LB (Nginx Session Sticky), the problem
still exists.
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Priority: Major
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3474:
---
Summary: Tableau 10.5 get malformed token (multi-query instance) (was: get
malformed token with Cluster Mode(multi-query))
> Tableau 10.5 get malformed token (multi-query instance)
> ---
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Priority: Major
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) get malformed token with Cluster Mode(multi-query)
[
https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570974#comment-16570974
]
Temple Zhou commented on KYLIN-3474:
[~albertoramon]
Oh, maybe you are right, I will check my LB.
> get malformed token with Cluster Mode(multi-query)
> --
>
> Key: KYLIN-3474
> URL: https://issues.apache.org/jira/browse/KYLIN-3474
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine, Security
>Affects Versions: v2.4.0
>Reporter: Temple Zhou
>Priority: Major
>
> I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
> Tableau Server will get the malformed token error when do query via restful
> API.
> However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
> error will be gone.
> Now, I just annotate the following
> lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and
> the Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
> {code:java}
> @Override
> public Authentication authenticate(Authentication authentication) throws
> AuthenticationException {
> byte[] hashKey = hf.hashString(authentication.getName() +
> authentication.getCredentials()).asBytes();
> String userKey = Arrays.toString(hashKey);
> // if (userService.isEvictCacheFlag()) {
> // userCache.invalidateAll();
> // userService.setEvictCacheFlag(false);
> // }
> Authentication authed = userCache.getIfPresent(userKey);
> if (null != authed) {
> SecurityContextHolder.getContext().setAuthentication(authed);
> } else {
> {code}
> I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3474) get malformed token with Cluster Mode(multi-query)
Temple Zhou created KYLIN-3474:
--
Summary: get malformed token with Cluster Mode(multi-query)
Key: KYLIN-3474
URL: https://issues.apache.org/jira/browse/KYLIN-3474
Project: Kylin
Issue Type: Bug
Components: Query Engine, Security
Affects Versions: v2.4.0
Reporter: Temple Zhou
I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the
Tableau Server will get the malformed token error when do query via restful API.
However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the
error will be gone.
Now, I just annotate the following
lines(org.apache.kylin.rest.security.KylinAuthenticationProvider 84-87) and the
Tableau 10.5 can run regularly with Kylin 2.4.0(multi-query).
{code:java}
@Override
public Authentication authenticate(Authentication authentication) throws
AuthenticationException {
byte[] hashKey = hf.hashString(authentication.getName() +
authentication.getCredentials()).asBytes();
String userKey = Arrays.toString(hashKey);
// if (userService.isEvictCacheFlag()) {
// userCache.invalidateAll();
// userService.setEvictCacheFlag(false);
// }
Authentication authed = userCache.getIfPresent(userKey);
if (null != authed) {
SecurityContextHolder.getContext().setAuthentication(authed);
} else {
{code}
I don't know if you tested the Kylin(multi-query) Cluster Mode with Tableau?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561731#comment-16561731
]
Temple Zhou commented on KYLIN-3460:
I found the issue had fixed with commit id 01df768
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555196#comment-16555196
]
Temple Zhou commented on KYLIN-3460:
[~Shaofengshi]
Hi Shaofeng, we are trying to use Tableau 2018.1 with Kylin 2.4 and many
problems, just like I mentioned above, have been discovered.
I'm not only willing to contribute a patch for this issue, I also want to help
Kylin to be compatible with Tableau 2018.1 :D
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v2.4.1, v2.5.0
>
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou reassigned KYLIN-3460:
--
Assignee: Temple Zhou
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Critical
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550624#comment-16550624
]
Temple Zhou commented on KYLIN-3460:
I have found the reason, this is because of the
DefaultQueryTransformer.PTN_HAVING_ESCAPE_FUNCTION.
With Kylin 2.1.0, the PTN_HAVING_ESCAPE_FUNCTION is
{code:java}
Pattern.compile("\\{fn" + "(.*?)" + "\\}", Pattern.CASE_INSENSITIVE)
{code}
with Kylin 2.4.0, the PTN_HAVING_ESCAPE_FUNCTION is
{code:java}
Pattern.compile("\\{fn" + SM + "(EXTRACT\\(.*?\\))" + "\\}",
Pattern.CASE_INSENSITIVE)
{code}
So, the Kylin 2.1.0 can transform the
{noformat}
SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL"
{noformat}
to
{noformat}
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
but the Kylin 2.4.0 can not.
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Priority: Critical
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550584#comment-16550584
]
Temple Zhou edited comment on KYLIN-3460 at 7/20/18 9:44 AM:
-
I also fount that the Kylin 2.1.0 can execute the query
{noformat}
SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL"
&
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
However, the Kylin 2.4.0 can only execute the query
{noformat}
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
When I execute the
{noformat}
SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL"
{noformat}
with Kylin 2.4.0, the error are as below:
{noformat}
>From line 1, column 8 to line 1, column 32: Function {fn CURRENT_TIMESTAMP} is
>not defined while executing SQL: SELECT \{fn CURRENT_TIMESTAMP(0)\} AS COL
>LIMIT 5
{noformat}
was (Author: temple.zhou):
I also fount that the Kylin 2.1.0 can execute the query
{noformat}
SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL"
&
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
However, the Kylin 2.4.0 can only execute the query
{noformat}
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
When I execute the SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL" with Kylin 2.4.0,
the error are as below:
{noformat}
>From line 1, column 8 to line 1, column 32: Function {fn CURRENT_TIMESTAMP} is
>not defined while executing SQL: SELECT \{fn CURRENT_TIMESTAMP(0)\} AS COL
>LIMIT 5
{noformat}
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Priority: Critical
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550584#comment-16550584
]
Temple Zhou commented on KYLIN-3460:
I also fount that the Kylin 2.1.0 can execute the query
{noformat}
SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL"
&
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
However, the Kylin 2.4.0 can only execute the query
{noformat}
SELECT CURRENT_TIMESTAMP(0) AS "COL"
{noformat}
When I execute the SELECT {fn CURRENT_TIMESTAMP(0)} AS "COL" with Kylin 2.4.0,
the error are as below:
{noformat}
>From line 1, column 8 to line 1, column 32: Function {fn CURRENT_TIMESTAMP} is
>not defined while executing SQL: SELECT \{fn CURRENT_TIMESTAMP(0)\} AS COL
>LIMIT 5
{noformat}
> {fn CURRENT_DATE()} parse error
> ---
>
> Key: KYLIN-3460
> URL: https://issues.apache.org/jira/browse/KYLIN-3460
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC, Query Engine
>Affects Versions: v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Priority: Critical
>
> Dear all,
> I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
> the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
> Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
> "2018-07-20 00:00:00"(the real current date). the logs are as below
> {noformat}
> Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {ts '2018-06-20 00:00:00'})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {ts '2018-07-20 00:00:00'})
>AND ("APP"."NAME" = 'xxx'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: true
> Duration: 0.06
> Project: app
> {noformat}
> However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
> are many Tableau reports get the same error due to the SQL contain {fn
> CURRENT_DATE()}
> [Tableau Error
> Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
> Besides, I found that the logs(2.3.1&2.4.0) are different from the
> logs(2.1.0), the logs(2.4.0) are as below
> {noformat}
> Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
> SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
>COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
> "usr_Calculation_267682710052360192_ok"
> FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> "APP_DAILY_ACTIVE_USER_PROPERTY_DI"
> LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
> "APP"."ID")
> INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
> ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
> WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
> {fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
>AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS
> TIMESTAMP) < {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
>AND ("APP"."NAME" = 'xxx')
>AND ("APP"."PLATFORM" = 'Android'))
> GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
> User: ADMIN
> Success: false
> Duration: 0.029
> Project: app
> {noformat}
> The problem that bothers me the most is that why the older version can parse
> the "fn CURRENT_DATE()" but the newer can't. :( :( :(
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3460:
---
Description:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver(2.1.0), and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to
"2018-07-20 00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
AND ("APP"."NAME" = 'xxx')
AND ("APP"."PLATFORM" = 'Android'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: false
Duration: 0.029
Project: app
{noformat}
The problem that bothers me the most is that why the older version can parse
the "fn CURRENT_DATE()" but the newer can't. :( :( :(
was:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver, and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to "2018-07-20
00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE
[jira] [Updated] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3460:
---
Description:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver, and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to "2018-07-20
00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
AND ("APP"."NAME" = 'xxx')
AND ("APP"."PLATFORM" = 'Android'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: false
Duration: 0.029
Project: app
{noformat}
The problem that bothers me the most is that why the older version can parse
the "fn CURRENT_DATE()" but the newer can't. :( :( :(
was:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver, and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to "2018-07-20
00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE
[jira] [Updated] (KYLIN-3460) {fn CURRENT_DATE()} parse error
[
https://issues.apache.org/jira/browse/KYLIN-3460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Temple Zhou updated KYLIN-3460:
---
Description:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver, and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to "2018-07-20
00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{fn TIMESTAMPADD(SQL_TSI_DAY,-29,{fn CURRENT_DATE()})})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {fn TIMESTAMPADD(SQL_TSI_DAY,1,{fn CURRENT_DATE()})})
AND ("APP"."NAME" = 'xxx')
AND ("APP"."PLATFORM" = 'Android'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: false
Duration: 0.029
Project: app
{noformat}
The problem that bothers me the most is that why the old version can parse the
was:
Dear all,
I'm facing a very critical problem. When I use Tableau 10.5 with Kylin 2.1.0,
the Tableau Server will send many SQL with "{fn CURRENT_DATE()}" via ODBC
Driver, and the Kylin Server 2.1.0 will parse the CURRENT_DATE() to "2018-07-20
00:00:00"(the real current date). the logs are as below
{noformat}
Query Id: 14e7b152-6be0-4539-8fd0-c28bbffa6b25
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{ts '2018-06-20 00:00:00'})
AND (CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP)
< {ts '2018-07-20 00:00:00'})
AND ("APP"."NAME" = 'xxx'))
GROUP BY "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV"
User: ADMIN
Success: true
Duration: 0.06
Project: app
{noformat}
However, recently, I upgrade my Kylin Server to 2.3.1 even 2.4.0, then there
are many Tableau reports get the same error due to the SQL contain {fn
CURRENT_DATE()}
[Tableau Error
Image|https://ws3.sinaimg.cn/large/006tKfTcgy1ftfbzshgs1j30fc0g2dhi.jpg]
Besides, I found that the logs(2.3.1&2.4.0) are different from the logs(2.1.0),
the logs(2.4.0) are as below
{noformat}
Query Id: b160f20c-84e4-4b81-bfe9-54ed22a723d6
SQL: SELECT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AV" AS "AV",
COUNT(DISTINCT "APP_DAILY_ACTIVE_USER_PROPERTY_DI"."MC") AS
"usr_Calculation_267682710052360192_ok"
FROM "ADL"."APP_DAILY_ACTIVE_USER_PROPERTY_DI"
"APP_DAILY_ACTIVE_USER_PROPERTY_DI"
LEFT JOIN "DIM"."APP" "APP" ON ("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."AC" =
"APP"."ID")
INNER JOIN "DIM"."CALENDAR" "CALENDAR" ON
("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" = "CALENDAR"."CAL_DT")
WHERE ((CAST("APP_DAILY_ACTIVE_USER_PROPERTY_DI"."ACTION_DT" AS TIMESTAMP) >=
{fn
