I think that making that configurable is a good thing but also it just seems an excessive amount of lines to begin with. Can anyone explain why this is getting called so many times? What value is there in calling it that much in such a short span of time (this is like 2-3 seconds).
And do we really need to be dumping these very similar stack traces to the job tracker logs AND the namenode logs? Also, in my specific case, I format the namenode, start up and all is good. Then i put about 7Gb of logs in via bin/hadoop dfs -put ... , ( dfs.replication is 1 ) wait for it to finish. stop-all.sh clear the logs start-all.sh and voilla I'm back to a million+ log lines on startup. -- - kate = masukomi http://weblog.masukomi.org/ On 9/26/07, Owen O'Malley <[EMAIL PROTECTED]> wrote: > > On Sep 26, 2007, at 9:02 AM, Ted Dunning wrote: > > > It is decidedly not OK in a near disk full situation. My > > experience was > > that HDFS filled the disk (almost) and then these logs made the > > situation > > nearly unrecoverable. > > If disk is tight, you probably should configure log4j to roll the > logs based on size rather than the default of daily. I just created > the jira HADOOP-1947 to make it easier to configure the appender and > the associated rolling strategy. > > -- Owen >
