Hi All, Trying to build cube but failing at building cube stage in process. I can see mapper after more attempts its getting failed.
Sending you the error message, please help me to resolve, attempt_1493717156723_1006_m_000000_0 100.00 FAILED hdfs://IALERT2HDP/kylin/kylin_metadata/kylin-ce08d124-99a1-4ee8-aa71-c02c8285dc1e/kylin_intermediate_AL_SFTY_RPT_CUBE_e00df75e_7cdb_4adc_af8d_ecb9db2bc8ab/000000_0:0+214404 > map pm10hdn02.ashokleyland.com:8042 logs <http://pm10hdn02.ashokleyland.com:8042/node/containerlogs/container_e35_1493717156723_1006_01_000002/hdfs> Mon May 15 11:17:53 +0550 2017 Mon May 15 11:21:32 +0550 2017 3mins, 38sec Error: java.io.IOException: Failed to build cube in mapper 0 at org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:145) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:149) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: java.lang.RuntimeException: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:143) ... 8 more Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: java.lang.RuntimeException: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:82) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: java.io.IOException: java.lang.RuntimeException: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:126) at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:73) at org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:80) ... 5 more Caused by: java.io.IOException: java.lang.RuntimeException: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.abort(DoggedCubeBuilder.java:194) at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.checkException(DoggedCubeBuilder.java:167) at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:114) ... 7 more Caused by: java.lang.RuntimeException: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$SplitThread.run(DoggedCubeBuilder.java:287) Caused by: java.io.IOException: 4 exceptions during in-mem cube build, cause set to the first, check log for more at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.throwExceptionIfAny(InMemCubeBuilder.java:228) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:184) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:135) at org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$SplitThread.run(DoggedCubeBuilder.java:282) Caused by: java.io.IOException: No space left on device at sun.nio.ch.FileDispatcherImpl.pwrite0(Native Method) at sun.nio.ch.FileDispatcherImpl.pwrite(FileDispatcherImpl.java:66) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:89) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:745) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:731) at org.apache.kylin.cube.inmemcubing.ConcurrentDiskStore$Writer$1.write(ConcurrentDiskStore.java:298) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at java.io.FilterOutputStream.close(FilterOutputStream.java:158) at org.apache.kylin.cube.inmemcubing.ConcurrentDiskStore$Writer.close(ConcurrentDiskStore.java:320) at org.apache.kylin.gridtable.GTBuilder.close(GTBuilder.java:53) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.scanAndAggregateGridTable(InMemCubeBuilder.java:448) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.aggregateCuboid(InMemCubeBuilder.java:397) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.buildCuboid(InMemCubeBuilder.java:389) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.access$300(InMemCubeBuilder.java:63) at org.apache.kylin.cube.inmemcubing.InMemCubeBuilder$CuboidTaskThread.run(InMemCubeBuilder.java:268) Suppressed: java.io.IOException: No space left on device at sun.nio.ch.FileDispatcherImpl.pwrite0(Native Method) at sun.nio.ch.FileDispatcherImpl.pwrite(FileDispatcherImpl.java:66) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:89) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:745) at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:731) at org.apache.kylin.cube.inmemcubing.ConcurrentDiskStore$Writer$1.write(ConcurrentDiskStore.java:298) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.FilterOutputStream.close(FilterOutputStream.java:158) at java.io.FilterOutputStream.close(FilterOutputStream.java:159) ... 7 more Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143Regards, Suresh
