>From the source code of hbase HFileOutputFormat2.java, it tries to create
file in ³/tmp² (HDFS path); you may need have a need on your HDFS:
static void configurePartitioner(Job job, List<ImmutableBytesWritable>
splitPoints)
throws IOException {
// create the partitions file
FileSystem fs = FileSystem.get(job.getConfiguration());
Path partitionsPath = new Path("/tmp", "partitions_" +
UUID.randomUUID());
On 7/15/15, 11:07 PM, "alex schufo" <[email protected]> wrote:
>Yes, my installation was apparently corrupted.
>I created a tmp folder and the build first steps work but now step #10
>fails with
>
>java.lang.IllegalArgumentException: Can not create a Path from a null
>string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)
> at org.apache.hadoop.fs.Path.<init>(Path.java:135)
> at org.apache.hadoop.fs.Path.<init>(Path.java:89)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(
>HFileOutputFormat2.java:545)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalL
>oad(HFileOutputFormat2.java:394)
> at
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLo
>ad(HFileOutputFormat.java:88)
> at
>org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:89)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
>org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable
>.java:112)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChai
>nedExecutable.java:50)
> at
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
> at
>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defaul
>tScheduler.java:132)
> at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1145)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:615)
> at java.lang.Thread.run(Thread.java:744)
>
>
>On Wed, Jul 15, 2015 at 2:44 PM, Shi, Shaofeng <[email protected]> wrote:
>
>> No, no need; All metadata and cubes are in Hbase/hdfs with redundancy;
>> Local disk is just using for temp files;
>>
>> On 7/15/15, 8:35 PM, "alex schufo" <[email protected]> wrote:
>>
>> >Do I have to re-create completely new cubes?
>> >
>> >On Wed, Jul 15, 2015 at 1:34 PM, alex schufo <[email protected]>
>> wrote:
>> >
>> >> I am in a situation where the machine where Kylin was installed went
>> >>down.
>> >> The Hadoop and HBase clusters are still working normally with the
>>Kylin
>> >> metadata still there.
>> >>
>> >> I installed Kylin on a new machine and I was assuming that thanks to
>>the
>> >> metadata it could work as before.
>> >>
>> >> It worked to an extent: I can access Kylin, see my existing projects,
>> >>see
>> >> my previous cubes and jobs.
>> >>
>> >> However if I go to the query panel of a cube I cannot see the Table
>>and
>> >>in
>> >> the log I can see :
>> >>
>> >>
>> >> [http-bio-7070-exec-6]:[2015-07-15
>>
>>>>05:33:06,917][ERROR][org.apache.kylin.rest.controller.BasicController.h
>>>>an
>> >>dleError(BasicController.java:52)]
>> >> -
>> >>
>> >> java.lang.RuntimeException: java.io.IOException: No such file or
>> >>directory
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.query.schema.OLAPSchemaFactory.createTempOLAPJson(OLAP
>>>>Sc
>> >>hemaFactory.java:134)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.service.BasicService.getOLAPDataSource(BasicServi
>>>>ce
>> >>.java:103)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.service.QueryService.getMetadata(QueryService.jav
>>>>a:
>> >>285)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.service.QueryService.getMetadata(QueryService.jav
>>>>a:
>> >>104)
>> >>
>> >> at
>> >>
>> >>org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$
>> 4957273f.in
>> >>voke(<generated>)
>> >>
>> >> at
>>net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedIntercep
>>>>to
>> >>r.intercept(Cglib2AopProxy.java:618)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$844b396.ge
>>>>tM
>> >>etadata(<generated>)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.controller.QueryController.getMetadata(QueryContr
>>>>ol
>> >>ler.java:179)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.controller.QueryController$$FastClassByCGLIB$$fc0
>>>>39
>> >>d0b.invoke(<generated>)
>> >>
>> >> at
>>net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedIntercep
>>>>to
>> >>r.intercept(Cglib2AopProxy.java:618)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.rest.controller.QueryController$$EnhancerByCGLIB$$595f
>>>>f2
>> >>c9.getMetadata(<generated>)
>> >>
>> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>Method)
>> >>
>> >> at
>> >>
>>
>>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
>>>>va
>> >>:57)
>> >>
>> >> at
>> >>
>>
>>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
>>>>rI
>> >>mpl.java:43)
>> >>
>> >> at java.lang.reflect.Method.invoke(Method.java:606)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.method.support.InvocableHandlerMethod.invoke(In
>>>>vo
>> >>cableHandlerMethod.java:213)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.method.support.InvocableHandlerMethod.invokeFor
>>>>Re
>> >>quest(InvocableHandlerMethod.java:126)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.mvc.method.annotation.ServletInvocableH
>>>>an
>> >>dlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHan
>>>>dl
>> >>erAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.mvc.method.annotation.RequestMappingHan
>>>>dl
>> >>erAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter
>>>>.h
>> >>andle(AbstractHandlerMethodAdapter.java:80)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.DispatcherServlet.doDispatch(Dispatcher
>>>>Se
>> >>rvlet.java:923)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.DispatcherServlet.doService(DispatcherS
>>>>er
>> >>vlet.java:852)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.FrameworkServlet.processRequest(Framewo
>>>>rk
>> >>Servlet.java:882)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet
>>>>.j
>> >>ava:778)
>> >>
>> >> at
>>javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
>> >>
>> >> at
>>javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:303)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:208)
>> >>
>> >> at
>> >>
>>org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:241)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:208)
>> >>
>> >> at
>> >>
>>
>>>>com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(Abstra
>>>>ct
>> >>InstrumentedFilter.java:97)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:241)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:208)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:330)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.access.intercept.FilterSecurityInterce
>>>>pt
>> >>or.invoke(FilterSecurityInterceptor.java:118)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.access.intercept.FilterSecurityInterce
>>>>pt
>> >>or.doFilter(FilterSecurityInterceptor.java:84)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.access.ExceptionTranslationFilter.doFi
>>>>lt
>> >>er(ExceptionTranslationFilter.java:113)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.session.SessionManagementFilter.doFilt
>>>>er
>> >>(SessionManagementFilter.java:103)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.authentication.AnonymousAuthentication
>>>>Fi
>> >>lter.doFilter(AnonymousAuthenticationFilter.java:113)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.servletapi.SecurityContextHolderAwareR
>>>>eq
>> >>uestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.savedrequest.RequestCacheAwareFilter.d
>>>>oF
>> >>ilter(RequestCacheAwareFilter.java:45)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.authentication.www.BasicAuthentication
>>>>Fi
>> >>lter.doFilter(BasicAuthenticationFilter.java:150)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.authentication.ui.DefaultLoginPageGene
>>>>ra
>> >>tingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.authentication.AbstractAuthenticationP
>>>>ro
>>
>>>>cessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.authentication.logout.LogoutFilter.doF
>>>>il
>> >>ter(LogoutFilter.java:105)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.context.SecurityContextPersistenceFilt
>>>>er
>> >>.doFilter(SecurityContextPersistenceFilter.java:87)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy$VirtualFilterChain.do
>>>>Fi
>> >>lter(FilterChainProxy.java:342)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy.doFilterInternal(Filt
>>>>er
>> >>ChainProxy.java:192)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.security.web.FilterChainProxy.doFilter(FilterChainP
>>>>ro
>> >>xy.java:160)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(Del
>>>>eg
>> >>atingFilterProxy.java:346)
>> >>
>> >> at
>> >>
>>
>>>>org.springframework.web.filter.DelegatingFilterProxy.doFilter(Delegatin
>>>>gF
>> >>ilterProxy.java:259)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:241)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:208)
>> >>
>> >> at
>> >>
>>com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:205)
>> >>
>> >> at
>> >>
>>com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:241)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:208)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
>>>>ve
>> >>.java:220)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
>>>>ve
>> >>.java:122)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticato
>>>>rB
>> >>ase.java:504)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
>>>>a:
>> >>170)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
>>>>a:
>> >>103)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:95
>>>>0)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
>>>>.j
>> >>ava:116)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
>>>>42
>> >>1)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11
>>>>Pr
>> >>ocessor.java:1074)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Ab
>>>>st
>> >>ractProtocol.java:611)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.
>>>>ja
>> >>va:314)
>> >>
>> >> at
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >>
>> >> at
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThre
>>>>ad
>> >>.java:61)
>> >>
>> >> at java.lang.Thread.run(Thread.java:744)
>> >>
>> >> Caused by: java.io.IOException: No such file or directory
>> >>
>> >> at java.io.UnixFileSystem.createFileExclusively(Native
>>Method)
>> >>
>> >> at java.io.File.createNewFile(File.java:1006)
>> >>
>> >> at java.io.File.createTempFile(File.java:1989)
>> >>
>> >> at java.io.File.createTempFile(File.java:2040)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.query.schema.OLAPSchemaFactory.createTempOLAPJson(OLAP
>>>>Sc
>> >>hemaFactory.java:96)
>> >>
>> >> ... 85 more
>> >>
>> >> If I try to build an existing cube it fails on Step #2 with this
>> >>exception:
>> >>
>> >> pool-5-thread-2]:[2015-07-15
>>
>>>>05:08:25,256][ERROR][org.apache.kylin.job.common.MapReduceExecutable.do
>>>>Wo
>> >>rk(MapReduceExecutable.java:115)]
>> >> - error execute
>> >> MapReduceExecutable{id=d729fd24-bb4f-47d6-b895-1e01594c60b5-01,
>> >> name=Extract Fact Table Distinct Columns, state=RUNNING}
>> >>
>> >> java.io.IOException: No such file or directory
>> >>
>> >> at java.io.UnixFileSystem.createFileExclusively(Native
>>Method)
>> >>
>> >> at java.io.File.createNewFile(File.java:1006)
>> >>
>> >> at java.io.File.createTempFile(File.java:1989)
>> >>
>> >> at java.io.File.createTempFile(File.java:2040)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.hadoop.AbstractHadoopJob.attachKylinPropsAndMetada
>>>>ta
>> >>(AbstractHadoopJob.java:217)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinc
>>>>tC
>> >>olumnsJob.java:78)
>> >>
>> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> >>
>> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecuta
>>>>bl
>> >>e.java:112)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu
>>>>ta
>> >>ble.java:107)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultC
>>>>ha
>> >>inedExecutable.java:50)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecu
>>>>ta
>> >>ble.java:107)
>> >>
>> >> at
>> >>
>>
>>>>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Def
>>>>au
>> >>ltScheduler.java:132)
>> >>
>> >> at
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >>
>> >> at
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >>
>> >> at java.lang.Thread.run(Thread.java:744)
>> >>
>>
>>
