Unfortunately yes, However you can modity this limit value by setting config property "kylin.table.snapshot.max_mb" which's default value is 300MB. BTW, if your dimension table is too large, It maybe lead to OOM.
2015-11-07 17:00 GMT+08:00, Ji, Hao Wei Jeffery <[email protected]>: > Hi, > > I meet a problem in building the cube. Seems there's a limitation for size > of dimension tables, is there any way to tune the threshold? > > > java.lang.IllegalStateException: Table snapshot should be no greater than > 300 MB, but TableDesc [database=DEFAULT name=DIMCUSTOMERCURRENT] size is > 506677558 > at > org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:95) > at > org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:205) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:60) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41) > at > org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:52) > at > org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:62) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > result code:2 > > > > > ************************************************************************************** > This e-mail and any attachments thereto may contain confidential information > and/or information protected by intellectual property rights for the > exclusive attention of the intended addressees named above. If you have > received this transmission in error, please immediately notify the sender by > return e-mail and delete this message and its attachments. Unauthorized use, > copying or further full or partial distribution of this e-mail or its > contents is prohibited. > ************************************************************************************** >
