sometimes it work sometimes not work :
2019-08-28 12:50:08,691 ERROR [http-bio-7070-exec-4]
controller.BasicController:63 :
org.apache.kylin.rest.exception.InternalErrorException
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:399)
at
org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:355)
at
org.apache.kylin.rest.controller.CubeController.build(CubeController.java:344)
......
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.rest.controller.CubeController.checkBuildingSegment(CubeController.java:995)
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:394)
... 79 more
2019-08-28 12:50:14,308 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:246
: Broadcasting UPDATE, cube, KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA
2019-08-28 12:50:14,310 DEBUG [http-bio-7070-exec-7] util.CompressionUtils:71 :
Original: 161 bytes. Decompressed: 312 bytes. Time: 0
2019-08-28 12:50:14,311 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:246
: Broadcasting UPDATE, project_data, KYLIN_SYSTEM
2019-08-28 12:50:14,311 INFO [http-bio-7070-exec-7] service.CacheService:121 :
cleaning cache for project KYLIN_SYSTEM (currently remove nothing)
2019-08-28 12:50:14,311 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280
: Done broadcasting UPDATE, project_data, KYLIN_SYSTEM
2019-08-28 12:50:14,312 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280
: Done broadcasting UPDATE, cube, KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA
2019-08-28 12:50:14,321 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:246
: Broadcasting CREATE, execute_output, fc003104-b34b-babe-9191-a67cd3288021
2019-08-28 12:50:14,322 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280
: Done broadcasting CREATE, execute_output, fc003104-b34b-babe-9191-a67cd3288021
2019-08-28 12:50:14,325 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:246
: Broadcasting CREATE, execute, fc003104-b34b-babe-9191-a67cd3288021
2019-08-28 12:50:14,326 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280
: Done broadcasting CREATE, execute, fc003104-b34b-babe-9191-a67cd3288021
2019-08-28 12:50:15,312 INFO [Thread-1405] service.CubeService:645 : checking
keepCubeRetention
2019-08-28 12:50:15,312 DEBUG [Thread-1405] model.Segments:200 : Cube
KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA has 1 building segments
2019-08-28 12:50:15,312 DEBUG [Thread-1405] service.CubeService:693 : Not ready
for merge on cube KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA
2019-08-28 12:50:30,045 INFO [FetcherRunner 890686756-192]
threadpool.FetcherRunner:62 :
CubingJob{id=fc003104-b34b-babe-9191-a67cd3288021, name=BUILD CUBE -
KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA - 20190827160000_20190828040000 - GMT+08:00
2019-08-28 12:50:14, state=READY} prepare to schedule and its priority is 20
2019-08-28 12:50:30,045 INFO [FetcherRunner 890686756-192]
threadpool.FetcherRunner:66 :
CubingJob{id=fc003104-b34b-babe-9191-a67cd3288021, name=BUILD CUBE -
KYLIN_HIVE_METRICS_JOB_EXCEPTI
ON_QA - 20190827160000_20190828040000 - GMT+08:00 2019-08-28 12:50:14,
state=READY} scheduled
2019-08-28 12:50:30,046 INFO [Scheduler 1378697493 Job
fc003104-b34b-babe-9191-a67cd3288021-565] execution.AbstractExecutable:151 :
Executing AbstractExecutable (BUILD CUBE - KYLIN_HIVE_METRICS_JOB_EXCEPTION_QA
- 20190827160000_20190828040000 - GMT+08:00 2019-08-28 12:50:14)
2019-08-28 12:50:30,046 INFO [FetcherRunner 890686756-192]
threadpool.DefaultFetcherRunner:85 : Job Fetcher: 0 should running, 1 actual
running, 0 stopped, 1 ready, 1407 already succeed, 1 error, 0 discarded, 0
others
2019-08-28 12:50:30,048 INFO [Scheduler 1378697493 Job
fc003104-b34b-babe-9191-a67cd3288021-565] execution.ExecutableManager:471 : job
id:fc003104-b34b-babe-9191-a67cd3288021 from READY to RUNNING
2019-08-28
lk_hadoop
发件人："lk_hadoop"<[email protected]>
发送时间：2019-08-28 10:44
主题：回复: 回复: 回复: CubeBuildingCLI not work well
收件人："user"<[email protected]>,"lk_hadoop"<[email protected]>
抄送："dev"<[email protected]>
I've use metastore.sh removed /model_desc/membership_marking.json and
/model_desc/gaoji_sale_model.json and restarted kylin , but still can't build
systemcube automatically.
2019-08-28
lk_hadoop
发件人："lk_hadoop"<[email protected]>
发送时间：2019-08-28 09:21
主题：回复: 回复: 回复：CubeBuildingCLI not work well
收件人："user"<[email protected]>
抄送："dev"<[email protected]>
NowI have upgrade from kylin2.6.1 to kylin2.6.3 , I still face the issue of
building system cube.
I follow the tutorial use cron job to call system_cube_build.sh , eche time I
will get the error log ,and I can't see any build job from the web ui:
2019-08-27 20:20:11,909 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading
DataModelDesc from kylin_metadata(key='/model_desc')@kylin_metadata@hbase
...
2019-08-27 16:40:02,319 ERROR [http-bio-7070-exec-10]
controller.BasicController:63 :
org.apache.kylin.rest.exception.InternalErrorException
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:399)
at
org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:355)
at
org.apache.kylin.rest.controller.CubeController.build(CubeController.java:344)
...
Caused by: java.lang.NullPointerException
at
org.apache.kylin.rest.controller.CubeController.checkBuildingSegment(CubeController.java:995)
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:394)
... 80 more
2019-08-27 16:40:02,342 ERROR [http-bio-7070-exec-8]
controller.CubeController:398 :
java.lang.NullPointerException
at
org.apache.kylin.rest.controller.CubeController.checkBuildingSegment(CubeController.java:995)
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:394)
at
org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:355)
at
org.apache.kylin.rest.controller.CubeController.build(CubeController.java:344)
...
and I also can find some error from the system cube build's log :
2019-08-27 20:20:11,943 ERROR [main] cachesync.CachedCrudAssist:143 : Error
loading DataModelDesc at /model_desc/gaoji_sale_model.json
java.lang.IllegalStateException: Error loading DataModelDesc at
/model_desc/gaoji_sale_model.json
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadAt(CachedCrudAssist.java:168)
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadQuietlyAt(CachedCrudAssist.java:141)
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadAll(CachedCrudAssist.java:124)
at
org.apache.kylin.metadata.model.DataModelManager.init(DataModelManager.java:99)
at
org.apache.kylin.metadata.model.DataModelManager.<init>(DataModelManager.java:79)
....
Caused by: java.lang.IllegalStateException: No project found for model
gaoji_sale_model
at
org.apache.kylin.metadata.project.ProjectManager.getProjectOfModel(ProjectManager.java:384)
at
org.apache.kylin.metadata.model.DataModelManager$1.initEntityAfterReload(DataModelManager.java:89)
at
org.apache.kylin.metadata.model.DataModelManager$1.initEntityAfterReload(DataModelManager.java:86)
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadAt(CachedCrudAssist.java:159)
... 45 more
2019-08-27 20:20:11,952 ERROR [main] cachesync.CachedCrudAssist:143 : Error
loading DataModelDesc at /model_desc/membership_marking.json
java.lang.IllegalStateException: Error loading DataModelDesc at
/model_desc/membership_marking.json
...
Caused by: java.lang.IllegalStateException: No project found for model
membership_marking
at
org.apache.kylin.metadata.project.ProjectManager.getProjectOfModel(ProjectManager.java:384)
at
org.apache.kylin.metadata.model.DataModelManager$1.initEntityAfterReload(DataModelManager.java:89)
at
org.apache.kylin.metadata.model.DataModelManager$1.initEntityAfterReload(DataModelManager.java:86)
at
org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadAt(CachedCrudAssist.java:159)
... 45 more
all that models which cause error , I've deleted already,
why kylin still want to load it from meta_table ？
how can I delete it ?
is this the reason which cause the system cube build error ? but when I use
admin to reload metadata from the web ui , all the build jobs should be
executed will appear to the web ui and begin to run.
2019-08-28
lk_hadoop
发件人："lk_hadoop"<[email protected]>
发送时间：2019-04-08 19:24
主题：回复: 回复：CubeBuildingCLI not work well
收件人："user"<[email protected]>
抄送：
thanks for reply @Chao Long . I have confirmed that there no other crontab
tasks , and the error msg just print out when the crontab job called.
2019-04-08
lk_hadoop
发件人："Chao Long" <[email protected]>
发送时间：2019-04-08 14:47
主题：回复：CubeBuildingCLI not work well
收件人："user"<[email protected]>
抄送：
Hi,
CubeBuildingCLI will not call method in CubeController, the stack trace show
some exceptions throw from method CubeController.build(), which will be called
by rest api. So you may check if there are other running cube-building crontab
tasks.
------------------
Best Regards,
Chao Long
------------------ 原始邮件 ------------------
发件人: "lk_hadoop"<[email protected]>;
发送时间: 2019年3月25日(星期一) 下午5:20
收件人: "user"<[email protected]>;
主题: CubeBuildingCLI not work well
hi,all:
I'm using kylin 2.6.1 with CDH57, When calling CubeBuildingCLI the job will
not be scheduled,only when I click "reload metadata" on kylin's web , The job
can be scheduled.
and if CubeBuildingCLI was called from crontab , I can see error message
from the log :
2019-03-25 16:20:02,325 ERROR [http-bio-7070-exec-2]
controller.BasicController:63 :
org.apache.kylin.rest.exception.InternalErrorException
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:398)
at
org.apache.kylin.rest.controller.CubeController.rebuild(CubeController.java:354)
at
org.apache.kylin.rest.controller.CubeController.build(CubeController.java:343)
at sun.reflect.GeneratedMethodAccessor210.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:653)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1137)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.rest.controller.CubeController.checkBuildingSegment(CubeController.java:984)
at
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:393)
... 77 more
2019-03-25
lk_hadoop
