By the way, I already create two simple patches for this problem. One is handle NPE itself, the other one(KYLIN-1468) is remove DataModeDesc from projects if saveDataModelDesc throw exceptions.
Please review it when you have time. Thanks. -- Best regards, John Zhao Cell: +1 (408) 464-5225 On 3/4/16, 12:24 AM, "hongbin ma" <[email protected]> wrote: >@john, can show us how to reproduce the issue? or can you contribute a >patch on this? > >On Fri, Mar 4, 2016 at 3:53 PM, Li Yang <[email protected]> wrote: > >> Agree with Hao, both 1) and 2) make very good sense. Should update JIRA >> comments as well so these are incorporated into fix. >> >> On Fri, Mar 4, 2016 at 2:21 AM, Hao Chen <[email protected]> wrote: >> >> > A few additional comments: >> > 1) In REST api response, if the exception message is null like NPE, we'd >> > better display the exception class, otherwise kylin user have to >> > investigate the root cause from log. >> > 2) Control the cube metadata during creation is a good approach, but may >> we >> > disable the invalid cube or mark the cube status as invalid instead of >> > influencing all the other cubes while having problem with single cube? >> The >> > invalid cube status may be caused because metadata is bad, hbase is >> > unstable, underly hbase table/region is not found or inconsistent and so >> > on. >> > >> > Regards, >> > Hao >> > >> > On Thu, Mar 3, 2016 at 10:04 AM, Zhao, John <[email protected]> wrote: >> > >> > > That’s right, before we store the metadata, we need make sure it >> already >> > > passed all the check. >> > > >> > > -- >> > > Best regards, >> > > John Zhao >> > > >> > > Cell: +1 (408) 464-5225 >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > On 3/2/16, 6:59 PM, "hongbin ma" <[email protected]> wrote: >> > > >> > > >btw, is it leaving a inconsistent state in the metadata store (cube >> > > >instance exist but cube desc or model missing)? if it is, please >> > > elaborate. >> > > >In that case something needs improving int the cube create controller, >> > > etc. >> > > > >> > > >On Thu, Mar 3, 2016 at 10:56 AM, hongbin ma <[email protected]> >> > wrote: >> > > > >> > > >> as the stacktrace said: >> > > >> >> > > >> java.lang.IllegalStateException: Derived can only be defined on >> lookup >> > > >> table, >> > > >> >> > > >> you need to redefine the cube desc >> > > >> >> > > >> On Thu, Mar 3, 2016 at 6:44 AM, zhaoyunjiong (JIRA) < >> [email protected]> >> > > >> wrote: >> > > >> >> > > >>> zhaoyunjiong created KYLIN-1464: >> > > >>> ----------------------------------- >> > > >>> >> > > >>> Summary: >> > > org.apache.kylin.cube.CubeInstance.getDataModelDesc >> > > >>> throw java.lang.NullPointerException >> > > >>> Key: KYLIN-1464 >> > > >>> URL: >> > https://issues.apache.org/jira/browse/KYLIN-1464 >> > > >>> Project: Kylin >> > > >>> Issue Type: Bug >> > > >>> Reporter: zhaoyunjiong >> > > >>> Assignee: zhaoyunjiong >> > > >>> >> > > >>> >> > > >>> User report can't see models from Web UI. >> > > >>> After I check log, I found java.lang.NullPointerException >> > > >>> {quote} >> > > >>> 2016-03-02 08:06:59,645 ERROR [http-bio-7070-exec-22] >> > > >>> controller.BasicController:44 : >> > > >>> java.lang.NullPointerException >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.cube.CubeInstance.getDataModelDesc(CubeInstance.java:128) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.metadata.MetadataManager.getModels(MetadataManager.java:326) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService.listAllModels(ModelService.java:55) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService.getModels(ModelService.java:74) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService$$FastClassByCGLIB$$eb0205be.invoke(<generated>) >> > > >>> at >> > net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService$$EnhancerByCGLIB$$2e750d6d.getModels(<generated>) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.controller.ModelController.getModels(ModelController.java:71) >> > > >>> at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown >> > > Source) >> > > >>> at >> > > >>> >> > > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > > >>> at java.lang.reflect.Method.invoke(Method.java:606) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) >> > > >>> {quote} >> > > >>> >> > > >>> And before that we can see from log why it happens. >> > > >>> {quote} >> > > >>> 2016-03-02 08:06:59,238 INFO [http-bio-7070-exec-22] >> > > >>> cube.CubeDescManager:197 : Reloading Cube Metadata from folder >> > > >>> _kylin_metadata_qa2(key='/cube_desc')@_kylin_metadata_qa2@hbase >> > > >>> 2016-03-02 08:06:59,632 ERROR [http-bio-7070-exec-22] >> > > >>> cube.CubeDescManager:207 : Error loading cube desc >> > > /cube_desc/test4.json >> > > >>> java.lang.IllegalStateException: Derived can only be defined on >> > lookup >> > > >>> table, cube CubeDesc [name=test4], DimensionDesc >> > > >>> [name=DEFAULT.SLNG_CATEG_META_LVL2_DERIVED, join=null, >> > hierarchy=null, >> > > >>> table=DEFAULT.SLNG_CATEG_META_LVL2, column=null, >> > > derived=[CATEG_LVL2_NAME, >> > > >>> META_CATEG_NAME]] >> > > >>> at >> > > >>> >> > org.apache.kylin.cube.model.DimensionDesc.init(DimensionDesc.java:111) >> > > >>> at >> > org.apache.kylin.cube.model.CubeDesc.init(CubeDesc.java:461) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.cube.CubeDescManager.loadCubeDesc(CubeDescManager.java:134) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.cube.CubeDescManager.reloadAllCubeDesc(CubeDescManager.java:205) >> > > >>> at >> > > >>> >> org.apache.kylin.cube.CubeDescManager.<init>(CubeDescManager.java:94) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.cube.CubeDescManager.getInstance(CubeDescManager.java:74) >> > > >>> at >> > > >>> >> > org.apache.kylin.cube.CubeInstance.getDescriptor(CubeInstance.java:123) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.cube.CubeInstance.getDataModelDesc(CubeInstance.java:128) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.metadata.MetadataManager.getModels(MetadataManager.java:326) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService.listAllModels(ModelService.java:55) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService.getModels(ModelService.java:74) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService$$FastClassByCGLIB$$eb0205be.invoke(<generated>) >> > > >>> at >> > net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.service.ModelService$$EnhancerByCGLIB$$2e750d6d.getModels(<generated>) >> > > >>> at >> > > >>> >> > > >> > >> org.apache.kylin.rest.controller.ModelController.getModels(ModelController.java:71) >> > > >>> at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown >> > > Source) >> > > >>> at >> > > >>> >> > > >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > > >>> at java.lang.reflect.Method.invoke(Method.java:606) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) >> > > >>> at >> > > >>> >> > > >> > >> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java: >> > > >>> {quote} >> > > >>> >> > > >>> >> > > >>> >> > > >>> -- >> > > >>> This message was sent by Atlassian JIRA >> > > >>> (v6.3.4#6332) >> > > >>> >> > > >> >> > > >> >> > > >> >> > > >> -- >> > > >> Regards, >> > > >> >> > > >> *Bin Mahone | 马洪宾* >> > > >> Apache Kylin: http://kylin.io >> > > >> Github: https://github.com/binmahone >> > > >> >> > > > >> > > > >> > > > >> > > >-- >> > > >Regards, >> > > > >> > > >*Bin Mahone | 马洪宾* >> > > >Apache Kylin: http://kylin.io >> > > >Github: https://github.com/binmahone >> > > >> > >> > > > >-- >Regards, > >*Bin Mahone | 马洪宾* >Apache Kylin: http://kylin.io >Github: https://github.com/binmahone
