The filename "masters" lists the machines on which you might want to run the Secondary Namenode. The Secondary Namenode periodically merges the HDFS transaction log with the file system image. If you do not run a Secondary Namenode, then the NameNode does this functionality when it starts up.
Hope this helps, dhruba -----Original Message----- From: Emmanuel JOKE [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 13, 2007 6:01 AM To: [email protected] Subject: Hadoop Log4j ? It seems I'm having a lot of trouble trying to configure hadoop on one machine. I've followed the wiki tutorial and I've configured every thing on 1 machine. I tried to start hadoop using start-all.sh and it works. I've the following output: starting namenode, logging to /data/sengine/search/logs/hadoop- nutch-namenode-node-n1.out localhost: starting datanode, logging to /data/sengine/search/logs/hadoop- nutch-datanode-node-n1.out cat: /data/sengine/search/bin/../conf/masters: No such file or directory starting jobtracker, logging to /data/sengine/search/logs/hadoop- nutch-jobtracker-node-n1.out localhost: starting tasktracker, logging to /data/sengine/search/logs/hadoop- nutch-tasktracker-node-n1.out However do you have any idea why i've an error about a file name masters ? In the task manager i can see 3 process up and running: 1=>> /usr/local/java/bin/java -Xmx1000m - Dhadoop.log.dir=/data/sengine/search/logs - Dhadoop.log.file=hadoop-nutch-namenode-node-n1.log - Dhadoop.home.dir=/data/sengine/search - Dhadoop.id.str=nutch-Dhadoop.root.logger=INFO,console - Djava.library.path=/data/sen... 2=>> /usr/local/java/bin/java -Xmx1000m - Dhadoop.log.dir=/data/sengine/search/logs - Dhadoop.log.file=hadoop-nutch-jobtracker-node-n1.log - Dhadoop.home.dir=/data/sengine/search - Dhadoop.id.str=nutch-Dhadoop.root.logger=INFO ,console - Djava.library.path=/data/sengine/s... 3=>> /usr/local/java/bin/java -Xmx1000m - Dhadoop.log.dir=/data/sengine/search/logs - Dhadoop.log.file=hadoop-nutch-tasktracker-node-n1.log - Dhadoop.home.dir=/data/sengine/search - Dhadoop.id.str=nutch-Dhadoop.root.logger=INFO,console - Djava.library.path=/data ... I tried to start to crawl a website and I had the following error: $ bin/nutch crawl urls/nutch -dir crawl /usr/local/java/bin/java -Xmx512m - Dhadoop.log.dir=/data/sengine/search/logs -Dhadoop.log.file=hadoop.log - Djava.library.path=/data/sengine/search/lib/native/Linux-i386-32 -classpath /data/sengine/search/conf:/usr/local/java/lib/tools.jar:/data/sengine/search /build:/data/sengine/search/build/nutch- 1.0-dev.job:/data/sengine/search/build/test/classes:/data/sengine/search/nut ch-*.job:/data/sengine/search/lib/commons-cli-2.0-SNAPSHOT.jar:/data/sengine /search/lib/commons-codec-1.3.jar:/data/sengine/search/lib/commons-httpclien t-3.0.1.jar:/data/sengine/search/lib/commons-lang-2.1.jar:/data/sengine/sear ch/lib/commons-logging-1.0.4.jar:/data/sengine/search/lib/commons-logging-ap i-1.0.4.jar:/data/sengine/search/lib/hadoop-0.12.2-core.jar:/data/sengine/se arch/lib/jakarta-oro-2.0.7.jar:/data/sengine/search/lib/jets3t-0.5.0.jar:/da ta/sengine/search/lib/jetty-5.1.4.jar:/data/sengine/search/lib/junit-3.8.1.j ar:/data/sengine/search/lib/log4j-1.2.13.jar:/data/sengine/search/lib/lucene -core-2.1.0.jar:/data/sengine/search/lib/lucene-misc-2.1.0.jar:/data/sengine /search/lib/servlet-api.jar:/data/sengine/search/lib/taglibs-i18n.jar:/data/ sengine/search/lib/xerces-2_6_2-apis.jar:/data/sengine/search/lib/xerces-2_6 _2.jar:/data/sengine/search/lib/jetty-ext/ant.jar:/data/sengine/search/lib/j etty-ext/commons-el.jar:/data/sengine/search/lib/jetty-ext/jasper-compiler.j ar:/data/sengine/search/lib/jetty-ext/jasper-runtime.jar:/data/sengine/searc h/lib/jetty-ext/jsp-api.jar org.apache.nutch.crawl.Crawl urls/nutch -dir crawl crawl started in: crawl rootUrlDir = urls/nutch threads = 10 depth = 5 Injector: starting Injector: crawlDb: crawl/crawldb Injector: urlDir: urls/nutch Injector: Converting injected urls to crawl db entries. task_0002_m_000000_0: log4j:ERROR setFile(null,true) call failed. task_0002_m_000000_0: java.io.FileNotFoundException: /data/sengine/search/logs (Is a directory) task_0002_m_000000_0: at java.io.FileOutputStream.openAppend(Native Method) task_0002_m_000000_0: at java.io.FileOutputStream.<init>( FileOutputStream.java:177) task_0002_m_000000_0: at java.io.FileOutputStream .<init>( FileOutputStream.java:102) task_0002_m_000000_0: at org.apache.log4j.FileAppender.setFile( FileAppender.java:289) task_0002_m_000000_0: at org.apache.log4j.FileAppender.activateOptions( FileAppender.java :163) task_0002_m_000000_0: at org.apache.log4j.DailyRollingFileAppender.activateOptions( DailyRollingFileAppender.java:215) task_0002_m_000000_0: at org.apache.log4j.config.PropertySetter.activate( PropertySetter.java :256) task_0002_m_000000_0: at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java :132) task_0002_m_000000_0: at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96) task_0002_m_000000_0: at org.apache.log4j.PropertyConfigurator.parseAppender( PropertyConfigurator.java:654) task_0002_m_000000_0: at org.apache.log4j.PropertyConfigurator.parseCategory( PropertyConfigurator.java:612) task_0002_m_000000_0: at org.apache.log4j.PropertyConfigurator.configureRootCategory( PropertyConfigurator.java:509) task_0002_m_000000_0: at org.apache.log4j.PropertyConfigurator.doConfigure (PropertyConfigurator.java :415) task_0002_m_000000_0: at org.apache.log4j.PropertyConfigurator.doConfigure (PropertyConfigurator.java:441) task_0002_m_000000_0: at org.apache.log4j.helpers.OptionConverter.selectAndConfigure( OptionConverter.java :468) task_0002_m_000000_0: at org.apache.log4j.LogManager.<clinit>( LogManager.java:122) task_0002_m_000000_0: at org.apache.log4j.Logger.getLogger(Logger.java :104) task_0002_m_000000_0: at org.apache.commons.logging.impl.Log4JLogger.getLogger (Log4JLogger.java:229) task_0002_m_000000_0: at org.apache.commons.logging.impl.Log4JLogger .<init>(Log4JLogger.java:65) task_0002_m_000000_0: at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) task_0002_m_000000_0: at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:39) task_0002_m_000000_0: at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java :27) task_0002_m_000000_0: at java.lang.reflect.Constructor.newInstance( Constructor.java:494) task_0002_m_000000_0: at org.apache.commons.logging.impl.LogFactoryImpl.newInstance( LogFactoryImpl.java:529) task_0002_m_000000_0: at org.apache.commons.logging.impl.LogFactoryImpl.getInstance( LogFactoryImpl.java:235) task_0002_m_000000_0: at org.apache.commons.logging.LogFactory.getLog( LogFactory.java:370) task_0002_m_000000_0: at org.apache.hadoop.mapred.TaskTracker .<clinit>( TaskTracker.java:82) task_0002_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main( TaskTracker.java:1423) task_0002_m_000000_0: log4j:ERROR Either File or DatePattern options are not set for appender [DRFA]. task_0002_m_000000_1: log4j:ERROR setFile(null,true) call failed. task_0002_m_000000_1: java.io.FileNotFoundException: /data/sengine/search/logs (Is a directory) task_0002_m_000000_1: at java.io.FileOutputStream.openAppend (Native Method) task_0002_m_000000_1: at java.io.FileOutputStream.<init>( FileOutputStream.java:177) task_0002_m_000000_1: at java.io.FileOutputStream.<init>( FileOutputStream.java:102) task_0002_m_000000_1: at org.apache.log4j.FileAppender.setFile( FileAppender.java:289) task_0002_m_000000_1: at org.apache.log4j.FileAppender.activateOptions( FileAppender.java:163) task_0002_m_000000_1: at org.apache.log4j.DailyRollingFileAppender.activateOptions ( DailyRollingFileAppender.java:215) task_0002_m_000000_1: at org.apache.log4j.config.PropertySetter.activate( PropertySetter.java:256) task_0002_m_000000_1: at org.apache.log4j.config.PropertySetter.setProperties (PropertySetter.java :132) task_0002_m_000000_1: at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96) task_0002_m_000000_1: at org.apache.log4j.PropertyConfigurator.parseAppender( PropertyConfigurator.java :654) task_0002_m_000000_1: at org.apache.log4j.PropertyConfigurator.parseCategory( PropertyConfigurator.java:612) task_0002_m_000000_1: at org.apache.log4j.PropertyConfigurator.configureRootCategory( PropertyConfigurator.java :509) task_0002_m_000000_1: at org.apache.log4j.PropertyConfigurator.doConfigure (PropertyConfigurator.java:415) task_0002_m_000000_1: at org.apache.log4j.PropertyConfigurator.doConfigure (PropertyConfigurator.java :441) task_0002_m_000000_1: at org.apache.log4j.helpers.OptionConverter.selectAndConfigure( OptionConverter.java:468) task_0002_m_000000_1: at org.apache.log4j.LogManager.<clinit>( LogManager.java:122) task_0002_m_000000_1: at org.apache.log4j.Logger.getLogger(Logger.java :104) task_0002_m_000000_1: at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229) task_0002_m_000000_1: at org.apache.commons.logging.impl.Log4JLogger.<init>( Log4JLogger.java:65) task_0002_m_000000_1: at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) task_0002_m_000000_1: at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java :39) task_0002_m_000000_1: at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java:27) task_0002_m_000000_1: at java.lang.reflect.Constructor.newInstance( Constructor.java :494) task_0002_m_000000_1: at org.apache.commons.logging.impl.LogFactoryImpl.newInstance( LogFactoryImpl.java:529) task_0002_m_000000_1: at org.apache.commons.logging.impl.LogFactoryImpl.getInstance( LogFactoryImpl.java :235) task_0002_m_000000_1: at org.apache.commons.logging.LogFactory.getLog( LogFactory.java:370) task_0002_m_000000_1: at org.apache.hadoop.mapred.TaskTracker.<clinit>( TaskTracker.java:82) task_0002_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main( TaskTracker.java:1423) task_0002_m_000000_1: log4j:ERROR Either File or DatePattern options are not set for appender [DRFA]. Any idea why I've this error ? I confirm that my log4j properties file is well defined and is located in conf folder. Thanks in advance for your help Cheers E
