Which step throws this error? From the log, it seems hadoop is opening local
file. Please check the default FileSystem of HDFS and also check the hadoop
job's memory configuration. There might be some hint.
________________________________
发件人: ashishjainm <[email protected]>
发送时间: 2017年4月14日 13:53:58
收件人: [email protected]
主题: Kylin Build Cube failing with the following error
Hello Team,
I am new to Kylin and just created the sample cube. however, when i create a
model with my own data and this is a very small data set with 10 rows in the
fact table. The process is failing with the following error message. I am
using the CDH quickstart VM.
May be this is a Yarn related issue and i wanted to know an optimum
configuration that i need to set for me to be able to build a sample cube of
my own.
82252_0024_01_000003/meta/cube_desc
2017-04-13 22:42:34,902 INFO [main]
org.apache.kylin.metadata.project.ProjectManager: Initializing
ProjectManager with metadata url
/yarn/nm/usercache/root/appcache/application_1492137382252_0024/container_1492137382252_0024_01_000003/meta
2017-04-13 22:42:35,008 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: Checking custom measure types
from kylin config
2017-04-13 22:42:35,011 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering
COUNT_DISTINCT(hllc), class
org.apache.kylin.measure.hllc.HLLCMeasureType$Factory
2017-04-13 22:42:35,028 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering
COUNT_DISTINCT(bitmap), class
org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory
2017-04-13 22:42:35,038 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering TOP_N(topn), class
org.apache.kylin.measure.topn.TopNMeasureType$Factory
2017-04-13 22:42:35,042 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering RAW(raw), class
org.apache.kylin.measure.raw.RawMeasureType$Factory
2017-04-13 22:42:35,046 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering
EXTENDED_COLUMN(extendedcolumn), class
org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory
2017-04-13 22:42:35,049 INFO [main]
org.apache.kylin.measure.MeasureTypeFactory: registering
PERCENTILE(percentile), class
org.apache.kylin.measure.percentile.PercentileMeasureType$Factory
2017-04-13 22:42:35,155 INFO [main] org.apache.kylin.cube.CubeDescManager:
Loaded 1 Cube(s)
2017-04-13 22:42:35,159 INFO [main] org.apache.kylin.cube.CubeManager:
Reloaded cube spend_cube being CUBE[name=spend_cube] having 1 segments
2017-04-13 22:42:35,159 INFO [main] org.apache.kylin.cube.CubeManager:
Loaded 1 cubes, fail on 0 cubes
2017-04-13 22:42:38,384 INFO [main] org.apache.hadoop.mapred.MapTask:
Starting flush of map output
2017-04-13 22:42:41,220 FATAL [main] org.apache.hadoop.mapred.YarnChild:
Error running child : java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
at java.lang.StringCoding.encode(StringCoding.java:344)
at java.lang.String.getBytes(String.java:916)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:216)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209)
at
org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:295)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:281)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:901)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:863)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1843)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1508)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2016)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2017-04-13 22:42:41,332 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
2017-04-13 22:42:41,333 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
stopped.
2017-04-13 22:42:41,333 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
shutdown complete.
--
View this message in context:
http://apache-kylin.74782.x6.nabble.com/Kylin-Build-Cube-failing-with-the-following-error-tp7646.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
