Hi, Ed,
Funny you should choose just now to reply. I just solved the problem on my own system and was about to post what I found. This appears to be related to HADOOP-406: https://issues.apache.org/jira/browse/HADOOP-406 That appears to be why the child JVM fails to inherit hadoop.log.dir, etc. I applied the patch for that, and then added the following to mapred.child.java.opts in conf/hadoop-site.xml: [EMAIL PROTECTED]@ [EMAIL PROTECTED]@ I also made my HADOOP_LOG_DIR environment var an absolute path instead of a relative one, so that any inheriting processes don't care where they're running (that did seem to be a problem, too). I don't know if Chris Schneider's patch for HADOOP-406 will prove to be the "long-term" solution, but it certainly works for me. Thanks for following up! Doug e w wrote: > > Hi Doug, > > There was a discussion under the subject "log4j.properties bug (?)" a > couple > of weeks back. Please check it out. My (temporary) solution was to > hardwire > the "log4j.appender.DRFA.File" variable in log4j.properties to > hadoop.logand then all the fetcher output from all tasks gets written > there as well. > Not an elegant solution but it works for debugging purposes. The problem > seems to be to do with the environment variable ${hadoop.log.dir} not > being > set when log4j.properties is parsed and so nutch tries to write to a file > in > "/". > > -Ed > > On 8/22/06, Doug Cook <[EMAIL PROTECTED]> wrote: >> >> >> Hi, Ed- >> >> I'm seeing the same problem. If anyone has had a similar experience and >> solved it, please let me know. In the mean time, I'll keep investigating >> and >> post back if I figure out what's going wrong. >> >> This may or may not matter, but I'm running everything on a single MP >> machine w/o DFS. >> >> Doug >> >> >> e w wrote: >> > >> > Logging of the Fetcher output in 0.8-dev used to work (writing to the >> > corresponding tasktracker output log) but doesn't appear to any more >> with >> > the nightly build from a couple of weeks ago and also the one from last >> > night. >> > >> > I've enabled DEBUG for the first 4 logging properties in >> > conf/log4j.properties and also: >> > >> > <property> >> > <name>fetcher.verbose</name> >> > <value>true</value> >> > <description>If true, fetcher will log more verbosely.</description> >> > </property> >> > >> > in nutch-site.xml. >> > >> > Does anyone else have this problem? If not, can anyone suggest a likely >> > cause? Thanks, >> > >> > -Ed >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/Problem-with-logging-of-Fetcher-output-in-0.8-dev-tf1999249.html#a5914756 >> Sent from the Nutch - User forum at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/Problem-with-logging-of-Fetcher-output-in-0.8-dev-tf1999249.html#a5956385 Sent from the Nutch - User forum at Nabble.com. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
