What's the "hbase.client.keyvalue.maxsize" in your hbase configuration? Guess it is bigger than 10485760. Please get that value and then set "kylin.hbase.client.keyvalue.maxsize" conf/kylin.properteis to the same or less than hbase's. Kylin uses the later one to deciside whether put a resource to hbase or not;
2016-05-03 17:09 GMT+08:00 彭丽 <[email protected]>: > Hi, > > I meet a problem when the step of building dimension dictionary. Fact > table has 400000 records. 3 lookup tables have 200000 , 10000, 200 records > separately. > > > > java.lang.IllegalArgumentException: KeyValue size too large > > at > org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1413) > > at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:1035) > > at org.apache.hadoop.hbase.client.HTable.put(HTable.java:999) > > at > > org.apache.kylin.storage.hbase.HBaseResourceStore.putResourceImpl(HBaseResou > rceStore.java:244) > > at > > org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore. > java:210) > > at > org.apache.kylin.dict.DictionaryManager.save(DictionaryManager.java:378) > > at > > org.apache.kylin.dict.DictionaryManager.saveNewDict(DictionaryManager.java:1 > 99) > > at > > org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.jav > a:166) > > at > > org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.ja > va:293) > > at > org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:167) > > at > > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe > neratorCLI.java:51) > > at > > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGe > neratorCLI.java:42) > > at > > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob > .java:56) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > > at > > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellEx > ecutable.java:60) > > at > > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable > .java:114) > > at > > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChaine > dExecutable.java:50) > > at > > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable > .java:114) > > at > > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultS > cheduler.java:124) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 > 45) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 > 15) > > at java.lang.Thread.run(Thread.java:745) > > > > result code:2 > > > > Thanks, > > > > Laney. > > > > -- Best regards, Shaofeng Shi
