Try increasing your driver memory. Thanks Best Regards
On Thu, Apr 16, 2015 at 6:09 PM, sarath <[email protected]> wrote: > Hi, > > I'm trying to train an SVM on KDD2010 dataset (available from libsvm). But > I'm getting "java.lang.OutOfMemoryError: Java heap space" error. The > dataset > is really sparse and have around 8 million data points and 20 million > features. I'm using a cluster of 8 nodes (each with 8 cores and 64G RAM). > > I have used both Spark's SVMWithSGD and Liblinear's Spark implementation > and > I'm getting "java.lang.OutOfMemoryError: Java heap space" error for both. > > I have used following settings: > executor-memory - 60G > num-executors - 64 > And other default settings > > Also I tried increasing the number of partitions. And tried with reduced > dataset of half million data points. But I'm still getting the same error. > > Here is the stack trace for Spark's SVMWithSGD: > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at > > org.apache.spark.mllib.optimization.GradientDescent$.runMiniBatchSGD(GradientDescent.scala:182) > at > > org.apache.spark.mllib.optimization.GradientDescent.optimize(GradientDescent.scala:107) > at > > org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm.run(GeneralizedLinearAlgorithm.scala:263) > at > > org.apache.spark.mllib.regression.GeneralizedLinearAlgorithm.run(GeneralizedLinearAlgorithm.scala:190) > at > org.apache.spark.mllib.classification.SVMWithSGD$.train(SVM.scala:201) > at > org.apache.spark.mllib.classification.SVMWithSGD$.train(SVM.scala:235) > at > org.apache.spark.mllib.classification.SVMWithSGD.train(SVM.scala) > at org.linearsvm.SVMClassifier.main(SVMClassifier.java:39) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:622) > at > > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) > at > org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) > at > org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > > > > > And the stack trace for Liblinear's Spark implementation : > > java.lang.OutOfMemoryError: Java heap space > at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) > at java.nio.ByteBuffer.allocate(ByteBuffer.java:329) > at > > org.apache.spark.network.BlockTransferService$$anon$1.onBlockFetchSuccess(BlockTransferService.scala:95) > at > > org.apache.spark.network.shuffle.RetryingBlockFetcher$RetryingBlockFetchListener.onBlockFetchSuccess(RetryingBlockFetcher.java:206) > at > > org.apache.spark.network.shuffle.OneForOneBlockFetcher$ChunkCallback.onSuccess(OneForOneBlockFetcher.java:72) > at > > org.apache.spark.network.client.TransportResponseHandler.handle(TransportResponseHandler.java:124) > at > > org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:93) > at > > org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:44) > at > > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > at > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) > at > > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) > at > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) > at > > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163) > at > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) > at > > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) > at > > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787) > at > > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130) > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) > at > > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) > at > > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at > > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) > at java.lang.Thread.run(Thread.java:701) > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-SVMWithSGD-java-lang-OutOfMemoryError-Java-heap-space-tp22524.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
