Hi, I am not sure the provided link matches my situation.
That is because, I am not trying to exclude any node but just trying to stop the NN act as a DataNode. Thanks & regards Arko On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee <arkoprovomukher...@gmail.com> wrote: > Thanks! Let me study and find out if it works for me. > Thanks & regards > Arko > > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee > <arkoprovomukher...@gmail.com> wrote: >> Hi, >> >> But the new location will also be the same NN physical server. >> So how does that help? >> >> I would like to clear up the space in the master by removing all the >> HDFS data to the slave nodes. >> >> Thanks & regards >> Arko >> >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <donta...@gmail.com> wrote: >>> No, copy this directory to the new location of your choice. >>> >>> Warm Regards, >>> Tariq >>> https://mtariq.jux.com/ >>> cloudfront.blogspot.com >>> >>> >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee >>> <arkoprovomukher...@gmail.com> wrote: >>>> >>>> Thanks! >>>> However, in that case, how do I handle the current data that is already in >>>> NN? >>>> Do I have to reformat HDFS? >>>> Thanks & regards >>>> Arko >>>> >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq <donta...@gmail.com> >>>> wrote: >>>> > Just don't run the DN daemon on that machine. Dedicate that machine to >>>> > NN. >>>> > Remove the hostname of the NN machine from the 'slaves' file. >>>> > >>>> > Warm Regards, >>>> > Tariq >>>> > https://mtariq.jux.com/ >>>> > cloudfront.blogspot.com >>>> > >>>> > >>>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee >>>> > <arkoprovomukher...@gmail.com> wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> Thanks for the help! >>>> >> >>>> >> However, I am still unsure about how to "turn off" the datanode >>>> >> feature on the NN. >>>> >> >>>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is pointed to >>>> >> a directory. >>>> >> >>>> >> Should I just comment out that property? What would happen to the >>>> >> current data that there is in the Master? Will it get removed >>>> >> automatically? >>>> >> >>>> >> Thanks & regards >>>> >> Arko >>>> >> >>>> >> >>>> >> >>>> >> On Wed, Feb 13, 2013 at 1:55 PM, Mohammad Tariq <donta...@gmail.com> >>>> >> wrote: >>>> >> > You can specify the logging level as specified by Charles. But >>>> >> > turning >>>> >> > logs >>>> >> > off is never a good idea. Logs are really helpful in problem >>>> >> > diagnosis, >>>> >> > which are eventual. >>>> >> > >>>> >> > Warm Regards, >>>> >> > Tariq >>>> >> > https://mtariq.jux.com/ >>>> >> > cloudfront.blogspot.com >>>> >> > >>>> >> > >>>> >> > On Thu, Feb 14, 2013 at 1:22 AM, Arko Provo Mukherjee >>>> >> > <arkoprovomukher...@gmail.com> wrote: >>>> >> >> >>>> >> >> Hi, >>>> >> >> >>>> >> >> Yeah, my NameNode is also seconding as a DataNode. >>>> >> >> >>>> >> >> I would like to "turn off" this feature. >>>> >> >> >>>> >> >> Request help regarding the same. >>>> >> >> >>>> >> >> Thanks & regards >>>> >> >> Arko >>>> >> >> >>>> >> >> On Wed, Feb 13, 2013 at 1:38 PM, Charles Baker <cba...@sdl.com> >>>> >> >> wrote: >>>> >> >> > Hi Arko. Sounds like you may be running a DataNode on the NameNode >>>> >> >> > which >>>> >> >> > is >>>> >> >> > not recommended practice. Normally, the only files the NN stores >>>> >> >> > are >>>> >> >> > the >>>> >> >> > image and edits files. It does not store any actual HDFS data. If >>>> >> >> > you >>>> >> >> > must >>>> >> >> > run a DN on the NN, try turning down the logging in >>>> >> >> > /conf/log4j.properties: >>>> >> >> > >>>> >> >> > #hadoop.root.logger=INFO,console >>>> >> >> > #hadoop.root.logger=WARN,console >>>> >> >> > hadoop.root.logger=ERROR,console >>>> >> >> > >>>> >> >> > Depending on the logging information you require, of course. >>>> >> >> > >>>> >> >> > -Chuck >>>> >> >> > >>>> >> >> > >>>> >> >> > -----Original Message----- >>>> >> >> > From: Arko Provo Mukherjee [mailto:arkoprovomukher...@gmail.com] >>>> >> >> > Sent: Wednesday, February 13, 2013 11:32 AM >>>> >> >> > To: hdfs-user@hadoop.apache.org >>>> >> >> > Subject: Managing space in Master Node >>>> >> >> > >>>> >> >> > Hello Gurus, >>>> >> >> > >>>> >> >> > I am managing a Hadoop Cluster to run some experiments. >>>> >> >> > >>>> >> >> > The issue I am continuously facing is that the Master Node runs >>>> >> >> > out >>>> >> >> > of >>>> >> >> > disk >>>> >> >> > space due to logs and data files. >>>> >> >> > >>>> >> >> > I can monitor and delete log files. However, I cannot delete the >>>> >> >> > HDFS >>>> >> >> > data. >>>> >> >> > >>>> >> >> > Thus, is there a way to force Hadoop not to save any HDFS data in >>>> >> >> > the >>>> >> >> > Master >>>> >> >> > Node? >>>> >> >> > >>>> >> >> > Then I can use my master to handle the metadata only and store the >>>> >> >> > logs. >>>> >> >> > >>>> >> >> > Thanks & regards >>>> >> >> > Arko >>>> >> >> > SDL Enterprise Technologies, Inc. - all rights reserved. The >>>> >> >> > information contained in this email may be confidential and/or >>>> >> >> > legally >>>> >> >> > privileged. It has been sent for the sole use of the intended >>>> >> >> > recipient(s). >>>> >> >> > If you are not the intended recipient of this mail, you are hereby >>>> >> >> > notified >>>> >> >> > that any unauthorized review, use, disclosure, dissemination, >>>> >> >> > distribution, >>>> >> >> > or copying of this communication, or any of its contents, is >>>> >> >> > strictly >>>> >> >> > prohibited. If you have received this communication in error, >>>> >> >> > please >>>> >> >> > reply >>>> >> >> > to the sender and destroy all copies of the message. >>>> >> >> > Registered address: 201 Edgewater Drive, Suite 225, Wakefield, MA >>>> >> >> > 01880, >>>> >> >> > USA >>>> >> >> > >>>> >> > >>>> >> > >>>> > >>>> > >>> >>>
