Increase the memory size or split the file! On Thu, Jul 26, 2012 at 5:37 AM, pricila rr <[email protected]> wrote: > I'm trying to transform a file .txt of 1gb for seqfile and the error > occurs: OutOfMemoryError: Java heap space > How to solve? > I am using Hadoop and Mahout. > > $MAHOUT_HOME/bin/mahout seqdirectory --input '/home/usuario/Área de > Trabalho/Dados/base1.txt' --output '/home/usuario/Área de > Trabalho/seqFile/base1File' -c UTF-8 > MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath. > Warning: $HADOOP_HOME is deprecated. > > Running on hadoop, using /home/usuario/hadoop/bin/hadoop and > HADOOP_CONF_DIR=/home/usuario/hadoop/conf > MAHOUT-JOB: > /home/usuario/trunk/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar > Warning: $HADOOP_HOME is deprecated. > > 12/07/26 09:18:28 INFO common.AbstractJob: Command line arguments: > {--charset=[UTF-8], --chunkSize=[64], --endPhase=[2147483647], > --fileFilterClass=[org.apache.mahout.text.PrefixAdditionFilter], > --input=[/home/usuario/Área de Trabalho/Dados/base1.txt], --keyPrefix=[], > --output=[/home/usuario/Área de Trabalho/seqFile/base1File], > --startPhase=[0], --tempDir=[temp]} > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2882) > at > java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) > at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) > at java.lang.StringBuilder.append(StringBuilder.java:119) > at > org.apache.mahout.text.PrefixAdditionFilter.process(PrefixAdditionFilter.java:62) > at > org.apache.mahout.text.SequenceFilesFromDirectoryFilter.accept(SequenceFilesFromDirectoryFilter.java:90) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:845) > at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867) > at > org.apache.mahout.text.SequenceFilesFromDirectory.run(SequenceFilesFromDirectory.java:98) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at > org.apache.mahout.text.SequenceFilesFromDirectory.main(SequenceFilesFromDirectory.java:53) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-- Lance Norskog [email protected]
