[
https://issues.apache.org/jira/browse/KYLIN-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI closed KYLIN-1745.
-------------------------------
> Java heap space errors for all builds in Kylin 1.5.2
> ----------------------------------------------------
>
> Key: KYLIN-1745
> URL: https://issues.apache.org/jira/browse/KYLIN-1745
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v1.5.2
> Reporter: Nick Muerdter
> Assignee: Shaofeng SHI
> Priority: Major
> Fix For: v1.5.2.1
>
>
> When trying to build any cubes in Kylin 1.5.2, I get Java heap space errors
> on step 2 (Extract Fact Table Distinct Columns). I experience the same heap
> space error on my own cube, as well as using the sample cube data (from
> {{sample.sh}}).
> I'm testing on a fairly stock HDP-2.2.9.0-3393 installation. I've tried
> tweaking the memory settings, but even with the map memory tuned to allow
> 8GB, I still get the same heap space error even for the small sample data
> (where 8GB seems like it should be more than plenty).
> If I try the same build against the sample data on a fresh 1.5.1
> installation, the cube builds as expected, but with a fresh 1.5.2
> installation, I get these heap space errors on any build attempts.
> Here's the backtrace from the failed map job:
> {code}
> 2016-05-29 01:51:57,910 WARN [main]
> org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration:
> tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
> 2016-05-29 01:51:57,977 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period
> at 10 second(s).
> 2016-05-29 01:51:57,977 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> started
> 2016-05-29 01:51:57,984 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Executing with tokens:
> 2016-05-29 01:51:58,017 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind:
> mapreduce.job, Service: job_1464481246919_0079, Ident:
> (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@548e6d58)
> 2016-05-29 01:51:58,045 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-29 01:51:58,249 INFO [main] org.apache.hadoop.mapred.YarnChild:
> mapreduce.cluster.local.dir for child:
> /hadoop/yarn/local/usercache/api-umbrella/appcache/application_1464481246919_0079
> 2016-05-29 01:51:58,443 INFO [main]
> org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 2016-05-29 01:51:58,792 INFO [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
> Committer Algorithm version is 1
> 2016-05-29 01:51:58,801 INFO [main] org.apache.hadoop.mapred.Task: Using
> ResourceCalculatorProcessTree : [ ]
> 2016-05-29 01:51:59,078 INFO [main] org.apache.hadoop.mapred.MapTask:
> Processing split: org.apache.hive.hcatalog.mapreduce.HCatSplit@12dae582
> 2016-05-29 01:51:59,218 ERROR [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:983)
> at
> org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:401)
> at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}
> Any ideas? Let me know if I can supply any other details.
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
