Hi pengwen, I carefully checked that and I think it was caused by some dirtly data on fact table (fk but doesn't exist in lookup table); The invalid value "532" should be a FK column value. Since the lookup table is defined as "inner join" with the fact table, Kylin delegates building the dictionary from lookup table's PK; so if there is unmatched record, this error may happen. The discussion about this issue is in JIRA KYLIN-1855.
Regarding why in-mem building got failed but layer cubing got success, that is because the layer cubing tolerates some insane record (< 100 in one mapper), but in-mem building didn't. I opened JIRA KYLIN-2231 for tracking this. Thanks for the reporting! 2016-11-25 23:33 GMT+08:00 [email protected] <[email protected]>: > This attachment is diagnosis. > Tonight I tried to use the layer cubing (kylin.cube.algorithm:layer),I > found its ok. > > ------------------------------ > 袁鹏文 Pengwen Yuan > > > *From:* ShaoFeng Shi <[email protected]> > *Date:* 2016-11-25 17:31 > *To:* user <[email protected]> > *Subject:* Re: org.apache.kylin.dict.TrieDictionary: Not a valid value > Hi Pengwen, > > This question also be asked by other people before, but we wasn't able to > reproduce. Could you share your model and cube definition JSON? Thanks! > > 2016-11-25 15:41 GMT+08:00 [email protected] <[email protected]>: > >> hi all: >> >> When I build cube at "#15 Step Name: Build Cube", I got an error : >> >> >> 2016-11-25 14:17:23,724 INFO [main] >> org.apache.kylin.engine.mr.steps.InMemCuboidMapper: Reserve 1079 MB = 200 >> (MR reserve) + 879 (SYS reserve) >> 2016-11-25 14:17:23,732 INFO [pool-5-thread-1] >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build start >> 2016-11-25 14:17:23,758 INFO [pool-5-thread-1] >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Split #1 kickoff >> 2016-11-25 14:17:23,771 INFO [Thread-12] >> org.apache.kylin.cube.inmemcubing.InMemCubeBuilder: In Mem Cube Build start, >> CUBE_OLP_SGM_HOTELMONTHREPORT >> 2016-11-25 14:17:25,984 WARN [Thread-12] >> org.apache.kylin.common.util.MemoryBudgetController: Higher system avail >> 8886 MB in markLow() >> 2016-11-25 14:17:25,987 INFO [Thread-12] >> org.apache.kylin.cube.inmemcubing.InMemCubeBuilder: Calculating base cuboid >> 1272016-11-25 14:17:26,134 ERROR [Thread-12] >> org.apache.kylin.dict.TrieDictionary: Not a valid value: 532 >> 2016-11-25 14:17:27,136 ERROR [pool-5-thread-1] >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder: Dogged Cube Build error >> java.io.IOException: java.lang.IllegalArgumentException: Value not exists! >> at >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.abort(DoggedCubeBuilder.java:194) >> at >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.checkException(DoggedCubeBuilder.java:167) >> at >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:114) >> at >> org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:73) >> at >> org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:80) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.IllegalArgumentException: Value not exists! >> at >> org.apache.kylin.common.util.Dictionary.getIdFromValueBytes(Dictionary.java:162) >> at >> org.apache.kylin.dict.TrieDictionary.getIdFromValueImpl(TrieDictionary.java:167) >> at >> org.apache.kylin.common.util.Dictionary.getIdFromValue(Dictionary.java:98) >> >> >> My Kylin version : 1.5.4.1 >> >> hadoop : 2.6.0-cdh5.8.0 >> >> HBase : 1.2.0-cdh5.8.0 >> >> >> >> >> ------------------------------ >> 袁鹏文 Pengwen Yuan >> > > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- Best regards, Shaofeng Shi 史少锋
