Hi Alexis Fidalgo 1) I did not seen this query recently
2) you need to configure this property in slaves ( DataNode). *dfs.datanode.du.**reserved : *The number of bytes will be left free on the volumes used by the DataNodes. By Default,it's zero. For example if the disk capacity is 1TB and *dfs.datanode.du.**reserved * configured with* 100GB.*So DataNode will not use 100GB for block allocation,so this data can be used by nodemanager intermittent files, log files . May be you can plan your MR jobs accordingly this. Hope I cleared your doubts. On Sat, Feb 11, 2017 at 7:26 PM, Alexis Fidalgo <[email protected]> wrote: > Hello, i’ve tried to search archives (and google) regarding this issue but > had no luck. After some changes in our mapreduce code, it takes all the > available disk space on datanodes, before this change we hade no problem at > all, but since then, every few days, disks on datanodes (we have 4, all > with same configuration regarding disk, memory, processor, OS) becomes full > and we have no more mapreduce jobs completed. so i need to wipe datanodes > and format namenode and start all over again. > > Reading documentation i found this configuration for hdfs-site.xml > > <property> > <name>dfs.datanode.du.reserved</name> > <value>32212254720</value> > </property> > </configuration> > > Questions regarding this > > 1. is there any thread already on this issue to read and not to ask again > about it? > 2. if not 1, do i need to set up this property only on master or every > slave too? > 3. will this fix the problem or just avoid the disk become full but the MR > jobs will fail the same (no more space to work so we need to review our > code) > > > thanks in advance, sorry if im asking about an already discussed issue, i > just suscribed to the list. > > regards > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- --Brahma Reddy Battula
