Haile, do you have the cardinality statistics for your dimension columns? A) You can check that in “Tables” tab, clicking the database name, and then click your fact table, in the “Table schema” table, there is the “Cardinality” column. B) If no data shows there, you can trigger a MR job to calculate that: click “Admin” tab, then click “Calculate Cardinality” button, enter table name. It will run a job to scan the table, which may takes several or tens of minutes to finish. After the job is completed, refresh your web page recheck the cardinality.
Usually this error happens when a dimension column's cardinality is very high (e.g, > 10 Million); This need user’s attension that whether you really need such a col as dimension. If not, remove that from cube; If yes, you can choose not building dictionary for this column in the “Advanced” step (need provide the max_length for this column), then Kylin will not encode the values. On 12/1/15, 8:58 PM, "杨海乐" <[email protected]> wrote: >When I build a cube with nine dimensions, I get the error. > ><name>mapreduce.reduce.java.opts</name> ><value>-Xmx6400m</value> >・ I think the memory should be enough<javascript:void(0);>. > >2015-12-01 20:17:59,192 INFO [main] >org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge >called with 23 in-memory map-outputs and 14 on-disk map-outputs >2015-12-01 20:17:59,192 INFO [main] >org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Keeping 23 >segments, 838228446 bytes in memory for intermediate, on-disk merge >2015-12-01 20:17:59,194 INFO [main] >org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 14 >files, 26641700531 bytes from disk >2015-12-01 20:17:59,194 INFO [main] >org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 >segments, 0 bytes from memory into reduce >2015-12-01 20:17:59,194 INFO [main] org.apache.hadoop.mapred.Merger: >Merging 37 sorted segments >2015-12-01 20:17:59,199 INFO [main] org.apache.hadoop.mapred.Merger: >Merging 28 intermediate segments out of a total of 37 >2015-12-01 20:20:56,756 INFO [main] org.apache.hadoop.mapred.Merger: Down >to the last merge-pass, with 10 segments left of total size: 31993180203 >bytes >2015-12-01 20:20:56,832 INFO [main] >org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is >deprecated. Instead, use mapreduce.job.skiprecords >2015-12-01 20:20:56,839 INFO [main] >org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for meta >dir is >/data/slot0/tmp/nm-local-dir/usercache/kylin/appcache/application_14489551 >24847_0008/container_1448955124847_0008_01_000221/meta >2015-12-01 20:20:56,840 INFO [main] org.apache.kylin.common.KylinConfig: >Use >KYLIN_CONF=/data/slot0/tmp/nm-local-dir/usercache/kylin/appcache/applicati >on_1448955124847_0008/container_1448955124847_0008_01_000221/meta >2015-12-01 20:20:56,852 INFO [main] org.apache.kylin.cube.CubeManager: >Initializing CubeManager with config >/data/slot0/tmp/nm-local-dir/usercache/kylin/filecache/18/meta >2015-12-01 20:20:56,854 INFO [main] >org.apache.kylin.common.persistence.ResourceStore: Using metadata url >/data/slot0/tmp/nm-local-dir/usercache/kylin/filecache/18/meta for >resource store >2015-12-01 20:20:57,061 INFO [main] >org.apache.kylin.cube.CubeDescManager: Initializing CubeDescManager with >config /data/slot0/tmp/nm-local-dir/usercache/kylin/filecache/18/meta >2015-12-01 20:20:57,062 INFO [main] >org.apache.kylin.cube.CubeDescManager: Reloading Cube Metadata from >folder >/data/slot0/tmp/nm-local-dir/usercache/kylin/filecache/18/meta/cube_desc >2015-12-01 20:33:19,328 FATAL [main] org.apache.hadoop.mapred.YarnChild: >Error running child : java.lang.OutOfMemoryError: Java heap space > at java.util.HashMap.resize(HashMap.java:580) > at java.util.HashMap.addEntry(HashMap.java:879) > at java.util.HashMap.put(HashMap.java:505) > at java.util.HashSet.add(HashSet.java:217) > at >org.apache.kylin.job.hadoop.cube.FactDistinctColumnsReducer.reduce(FactDis >tinctColumnsReducer.java:74) > at >org.apache.kylin.job.hadoop.cube.FactDistinctColumnsReducer.reduce(FactDis >tinctColumnsReducer.java:48) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) > at >org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at >org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation. >java:1628) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > >2015-12-01 20:33:19,335 INFO [communication thread] >org.apache.hadoop.mapred.Task: Communication exception: >java.io.IOException: The client is stopped > at org.apache.hadoop.ipc.Client.getConnection(Client.java:1500) > at org.apache.hadoop.ipc.Client.call(Client.java:1438) > at org.apache.hadoop.ipc.Client.call(Client.java:1399) > at >org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.j >ava:244) > at com.sun.proxy.$Proxy7.statusUpdate(Unknown Source) > at org.apache.hadoop.mapred.Task$TaskReporter.run(Task.java:737) > at java.lang.Thread.run(Thread.java:745) >
