Arv Mistry wrote:
I'm using hadoop 0.17.0. Unfortunately I cant upgrade to 0.19.0 just
yet.
I'm trying to control the amount of extraneous files. I noticed there
are the following log files produced by hadoop;
On Slave
- userlogs (for each map/reduce job)
- stderr
- stdout
- syslog
- datanode .log file
- datanode .out file
- tasktracker .log file
- tasktracker .out file
On Master
- jobtracker .log file
- jobtracker .out file
- namenode .log file
- namenode .out file
- secondarynamenode .log file
- secondarynamenode .out file
- job .xml file
- history
- xml file for job
Does any body know of how to configure hadoop so I don't have to delete
these files manually? Or just so that they don't get created at all.
For the history files, I set hadoop.job.history.user.location to none in
the hadoop-site.xml file but I still get the history files created.
Setting hadoop.job.history.user.location to "none", makes only history
location specified for user. JT still has history location. History will
be cleanup after a month.
Userlogs will be cleaned up after "mapred.userlog.retain.hours", by
default , 24hrs.
Thanks
Amareshwari
Also I set in the log4j.properties the hadoop.root.logger=WARN but I
still see INFO messages in datanode,jobtracker etc logs
Thanks, in advance
Cheers Arv
