Ensure no jobs are running (cache limit is only for non-active cache files), check after a little while (takes sometime for the cleaner thread to kick in).
Arun On Apr 11, 2013, at 2:29 AM, <xia_y...@dell.com> <xia_y...@dell.com> wrote: > Hi Hemanth, > > For the hadoop 1.0.3, I can only find "local.cache.size" in file > core-default.xml, which is in hadoop-core-1.0.3.jar. It is not in > mapred-default.xml. > > I updated the value in file default.xml and changed the value to 500000. This > is just for my testing purpose. However, the folder > /tmp/hadoop-root/mapred/local/archive already goes more than 1G now. Looks > like it does not do the work. Could you advise if what I did is correct? > > <name>local.cache.size</name> > <value>500000</value> > > Thanks, > > Xia > > From: Hemanth Yamijala [mailto:yhema...@thoughtworks.com] > Sent: Monday, April 08, 2013 9:09 PM > To: user@hadoop.apache.org > Subject: Re: How to configure mapreduce archive size? > > Hi, > > This directory is used as part of the 'DistributedCache' feature. > (http://hadoop.apache.org/docs/r1.0.4/mapred_tutorial.html#DistributedCache). > There is a configuration key "local.cache.size" which controls the amount of > data stored under DistributedCache. The default limit is 10GB. However, the > files under this cannot be deleted if they are being used. Also, some > frameworks on Hadoop could be using DistributedCache transparently to you. > > So you could check what is being stored here and based on that lower the > limit of the cache size if you feel that will help. The property needs to be > set in mapred-default.xml. > > Thanks > Hemanth > > > On Mon, Apr 8, 2013 at 11:09 PM, <xia_y...@dell.com> wrote: > Hi, > > I am using hadoop which is packaged within hbase -0.94.1. It is hadoop 1.0.3. > There is some mapreduce job running on my server. After some time, I found > that my folder /tmp/hadoop-root/mapred/local/archive has 14G size. > > How to configure this and limit the size? I do not want to waste my space > for archive. > > Thanks, > > Xia > > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
