Thank you Shaofeng, The facts as you said, I'm going to serialize the build task to avoid this problem, a JIRA will be submitted later
-----邮件原件----- 发件人: ShaoFeng Shi [mailto:[email protected]] 发送时间: 2017年7月18日 17:49 收件人: dev <[email protected]> 抄送: user <[email protected]> 主题: Re: intermediate table not found when used the hive view Let me confirm one thing; the deleted hive table is an intermediate table for a view, "kylin_intermediate_DW_ZYB_V_DIM_SCENIC" in this case. The view "DW_ZYB_V_DIM_SCENIC" is a lookup table. This is a know issue; It impacts on using View as lookup, and run multiple builds in parallel. I had thought its occurancy will be very low, that's my fault. Please open a JIRA to track this. A not-perfect solution is, use different views for different cubes. But that may impact on query patterns. Another suggestion would be, using normal hive table as lookup, that doesn't have this issue. 2017-07-18 15:22 GMT+08:00 jianhui.yi <[email protected]>: > Hi all, > > I used the Hive view to build the cube, and multiple cube all use the > same view. When the build task is submitted at the same time, there > will be an error in the step "Extract Fact Table Distinct Columns". > > I noticed that the previous build task will delete the table in the > "Create Intermediate Flat Hive Table" step, which should be the next > task of building the cause of the error, how to solve this problem? > > Thanks! > > > > 2017-07-17 08:58:49,803 ERROR [Job 71a3e0e4-a713-44f8-a473- > 7496245857ec-302] > common.HadoopShellExecutable:65 : error execute > HadoopShellExecutable{id=71a3e0e4-a713-44f8-a473-7496245857ec-04, > name= > > Build Dimension Dictionary, state=RUNNING} > > java.lang.RuntimeException: cannot get HiveTableMeta > > at org.apache.kylin.source.hive.HiveTable.<init>(HiveTable. > java:50) > > at > org.apache.kylin.source.hive.HiveSource.createReadableTable( > HiveSource.java: > 49) > > at > org.apache.kylin.source.SourceFactory.createReadableTable( > SourceFactory.java > :42) > > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI. > decideInputTable(Dictionary > GeneratorCLI.java:118) > > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI. > processSegment(DictionaryGe > neratorCLI.java:63) > > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI. > processSegment(DictionaryGe > neratorCLI.java:54) > > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run( > CreateDictionaryJob > .java:66) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable. > doWork(HadoopShellEx > ecutable.java:63) > > at > org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable > .java:124) > > at > org.apache.kylin.job.execution.DefaultChainedExecutable. > doWork(DefaultChaine > dExecutable.java:64) > > at > org.apache.kylin.job.execution.AbstractExecutable. > execute(AbstractExecutable > .java:124) > > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$ > JobRunner.run(DefaultS > cheduler.java:142) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:11 > 45) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:6 > 15) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: org.apache.hive.service.cli.HiveSQLException: Error while > compiling statement: FAILED: SemanticException [Error 10001]: Table > not found kylin_intermediate_DW_ZYB_V_DIM_SCENIC > > at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:239) > > at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils. > java:225) > > at > org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244) > > at > org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:375 > ) > > at > org.apache.kylin.source.hive.BeelineHiveClient. > getHiveTableMeta(BeelineHiveC > lient.java:141) > > at org.apache.kylin.source.hive.HiveTable.<init>(HiveTable. > java:48) > > ... 16 more > > Caused by: org.apache.hive.service.cli.HiveSQLException: Error while > compiling statement: FAILED: SemanticException [Error 10001]: Table > not found kylin_intermediate_DW_ZYB_V_DIM_SCENIC > > at > org.apache.hive.service.cli.operation.Operation. > toSQLException(Operation.jav > a:385) > > at > org.apache.hive.service.cli.operation.SQLOperation. > prepare(SQLOperation.java > :136) > > at > org.apache.hive.service.cli.operation.SQLOperation. > runInternal(SQLOperation. > java:206) > > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:327 > ) > > at > org.apache.hive.service.cli.session.HiveSessionImpl. > executeStatementInternal > (HiveSessionImpl.java:424) > > at > org.apache.hive.service.cli.session.HiveSessionImpl. > executeStatementAsync(Hi > veSessionImpl.java:401) > > at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown > Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl > .java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at > org.apache.hive.service.cli.session.HiveSessionProxy. > invoke(HiveSessionProxy > .java:78) > > at > org.apache.hive.service.cli.session.HiveSessionProxy. > access$000(HiveSessionP > roxy.java:36) > > at > org.apache.hive.service.cli.session.HiveSessionProxy$1. > run(HiveSessionProxy. > java:63) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs( > UserGroupInformation.ja > va:1783) > > at > org.apache.hive.service.cli.session.HiveSessionProxy. > invoke(HiveSessionProxy > .java:59) > > at com.sun.proxy.$Proxy27.executeStatementAsync(Unknown > Source) > > at > org.apache.hive.service.cli.CLIService.executeStatementAsync( > CLIService.java > :260) > > at > org.apache.hive.service.cli.thrift.ThriftCLIService. > ExecuteStatement(ThriftC > LIService.java:500) > > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$ > ExecuteStatement.ge > tResult(TCLIService.java:1313) > > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$ > ExecuteStatement.ge > tResult(TCLIService.java:1298) > > at > org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > > at org.apache.thrift.TBaseProcessor.process( > TBaseProcessor.java:39) > > at > org.apache.hive.service.auth.TSetIpAddressProcessor. > process(TSetIpAddressPro > cessor.java:56) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run( > TThreadPoolServ > er.java:286) > > ... 3 more > > Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Table > not found kylin_intermediate_DW_ZYB_V_DIM_SCENIC > > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$ > QualifiedNameUtil.getTab > leName(DDLSemanticAnalyzer.java:1835) > > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer. > analyzeDescribeTable(DDL > SemanticAnalyzer.java:1985) > > at > org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer. > analyzeInternal(DDLSeman > ticAnalyzer.java:316) > > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer. > analyze(BaseSemanticAna > lyzer.java:223) > > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:488) > > at > org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1274) > > at > org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1261) > > at > org.apache.hive.service.cli.operation.SQLOperation. > prepare(SQLOperation.java > :134) > > ... 26 more > > -- Best regards, Shaofeng Shi 史少锋
