Hi Sam, 1. I am sorry I didn't quite get "how many methods could clean it correctly? ".
Since this directory contains only the temporary files it should get cleaned up after your jobs are over. If you still have unnecessary data present there you can delete it. Make sure no jobs are running while you clean this directory. 2. All the daemons use log4j and DailyRollingFileAppender, which does not have retention settings. You can change the behavior by changing the Appender of your choice in *log4j.properties* files under *HADOOP_HOME/conf*directory. The associated property is *log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender*. 3. You must never touch the content of these 2 directories. This the actual HDFS *data+metadata*, which you don't want to loose. You can't find more on log files here<http://blog.cloudera.com/blog/2010/11/hadoop-log-location-and-retention/> . HTH *Warm regards,* *Mohammad Tariq* *cloudfront.blogspot.com <http://cloudfront.blogspot.com>* On Wed, May 7, 2014 at 9:10 AM, sam liu <[email protected]> wrote: > Hi Experts, > > 1. The size of mapred.local.dir is big(30 GB), how many methods could > clean it correctly? > 2. For logs of NameNode/DataNode/JobTracker/TaskTracker, are they all > rolling type log? What's their max size? I can not find the specific > settings for them in log4j.properties. > 3. I find the size of dfs.name.dir and dfs.data.dir is very big now, are > there any files under them could be removed actually? Or all files under > the two folders could not be removed at all? > > Thanks! >
