[ 
https://issues.apache.org/jira/browse/KYLIN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265311#comment-15265311
 ] 

Shaofeng SHI commented on KYLIN-1077:
-------------------------------------

[~whenwin] the patch is merged to master; before that a few minor changes were 
made to make it work:
1) remove the job id from the intermediate table name, because the character 
"-" is not accepted in table name;
2) remove the table name prefix in TableDesc.java, because that doesn't solve 
the concurrent conflict issue: if two jobs runs in parallel and both uses the 
same view,  the second one will overwrite the name prefix on TableDesc, so 
there is still the possibility that the first one uses the second materialized 
table to build dict and snapshot.
3) I changed to use a fixed name for a lookup view's materialized table, e.g, 
kylin_intermediate_<view_name>; it will be re-created each time when build the 
cube; This also has the concurrent issue, but not sure how often it can happen, 
don't want to make that too complicated. We can re-visit it if seeing reporting.

> Support Hive View as Lookup Table
> ---------------------------------
>
>                 Key: KYLIN-1077
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1077
>             Project: Kylin
>          Issue Type: Improvement
>    Affects Versions: v1.1
>            Reporter: Francesco Palmiotto
>            Assignee: wangxianbin
>              Labels: newbie
>             Fix For: v1.5.2
>
>         Attachments: 0001-KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch, 
> KYLIN-1077-Support-Hive-View-as-Lookup-Table.patch
>
>
> If we use a view as a lookup table the cube building job fails when executing 
> the 3rd step (Build Dimension Dictionary) with this log:
> java.io.IOException: java.lang.NullPointerException
>       at 
> org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:72)
>       at 
> org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:202)
>       at 
> org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:166)
>       at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:52)
>       at 
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
>       at 
> org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52)
>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>       at 
> org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62)
>       at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
>       at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
>       at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
>       at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>       at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> result code:2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to