Stopping DN on one machine means excluding it. What exactly is the requirement? How many nodes do you have right now?
Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Thu, Feb 14, 2013 at 4:36 AM, Arko Provo Mukherjee < arkoprovomukher...@gmail.com> wrote: > Hi, > > I am not sure the provided link matches my situation. > > That is because, I am not trying to exclude any node but just trying > to stop the NN act as a DataNode. > > Thanks & regards > Arko > > On Wed, Feb 13, 2013 at 2:31 PM, Arko Provo Mukherjee > <arkoprovomukher...@gmail.com> wrote: > > Thanks! Let me study and find out if it works for me. > > Thanks & regards > > Arko > > > > On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee > > <arkoprovomukher...@gmail.com> wrote: > >> Hi, > >> > >> But the new location will also be the same NN physical server. > >> So how does that help? > >> > >> I would like to clear up the space in the master by removing all the > >> HDFS data to the slave nodes. > >> > >> Thanks & regards > >> Arko > >> > >> On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <donta...@gmail.com> > wrote: > >>> No, copy this directory to the new location of your choice. > >>> > >>> Warm Regards, > >>> Tariq > >>> https://mtariq.jux.com/ > >>> cloudfront.blogspot.com > >>> > >>> > >>> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee > >>> <arkoprovomukher...@gmail.com> wrote: > >>>> > >>>> Thanks! > >>>> However, in that case, how do I handle the current data that is > already in > >>>> NN? > >>>> Do I have to reformat HDFS? > >>>> Thanks & regards > >>>> Arko > >>>> > >>>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq <donta...@gmail.com> > >>>> wrote: > >>>> > Just don't run the DN daemon on that machine. Dedicate that machine > to > >>>> > NN. > >>>> > Remove the hostname of the NN machine from the 'slaves' file. > >>>> > > >>>> > Warm Regards, > >>>> > Tariq > >>>> > https://mtariq.jux.com/ > >>>> > cloudfront.blogspot.com > >>>> > > >>>> > > >>>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee > >>>> > <arkoprovomukher...@gmail.com> wrote: > >>>> >> > >>>> >> Hi, > >>>> >> > >>>> >> Thanks for the help! > >>>> >> > >>>> >> However, I am still unsure about how to "turn off" the datanode > >>>> >> feature on the NN. > >>>> >> > >>>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is pointed > to > >>>> >> a directory. > >>>> >> > >>>> >> Should I just comment out that property? What would happen to the > >>>> >> current data that there is in the Master? Will it get removed > >>>> >> automatically? > >>>> >> > >>>> >> Thanks & regards > >>>> >> Arko > >>>> >> > >>>> >> > >>>> >> > >>>> >> On Wed, Feb 13, 2013 at 1:55 PM, Mohammad Tariq < > donta...@gmail.com> > >>>> >> wrote: > >>>> >> > You can specify the logging level as specified by Charles. But > >>>> >> > turning > >>>> >> > logs > >>>> >> > off is never a good idea. Logs are really helpful in problem > >>>> >> > diagnosis, > >>>> >> > which are eventual. > >>>> >> > > >>>> >> > Warm Regards, > >>>> >> > Tariq > >>>> >> > https://mtariq.jux.com/ > >>>> >> > cloudfront.blogspot.com > >>>> >> > > >>>> >> > > >>>> >> > On Thu, Feb 14, 2013 at 1:22 AM, Arko Provo Mukherjee > >>>> >> > <arkoprovomukher...@gmail.com> wrote: > >>>> >> >> > >>>> >> >> Hi, > >>>> >> >> > >>>> >> >> Yeah, my NameNode is also seconding as a DataNode. > >>>> >> >> > >>>> >> >> I would like to "turn off" this feature. > >>>> >> >> > >>>> >> >> Request help regarding the same. > >>>> >> >> > >>>> >> >> Thanks & regards > >>>> >> >> Arko > >>>> >> >> > >>>> >> >> On Wed, Feb 13, 2013 at 1:38 PM, Charles Baker <cba...@sdl.com> > >>>> >> >> wrote: > >>>> >> >> > Hi Arko. Sounds like you may be running a DataNode on the > NameNode > >>>> >> >> > which > >>>> >> >> > is > >>>> >> >> > not recommended practice. Normally, the only files the NN > stores > >>>> >> >> > are > >>>> >> >> > the > >>>> >> >> > image and edits files. It does not store any actual HDFS > data. If > >>>> >> >> > you > >>>> >> >> > must > >>>> >> >> > run a DN on the NN, try turning down the logging in > >>>> >> >> > /conf/log4j.properties: > >>>> >> >> > > >>>> >> >> > #hadoop.root.logger=INFO,console > >>>> >> >> > #hadoop.root.logger=WARN,console > >>>> >> >> > hadoop.root.logger=ERROR,console > >>>> >> >> > > >>>> >> >> > Depending on the logging information you require, of course. > >>>> >> >> > > >>>> >> >> > -Chuck > >>>> >> >> > > >>>> >> >> > > >>>> >> >> > -----Original Message----- > >>>> >> >> > From: Arko Provo Mukherjee [mailto: > arkoprovomukher...@gmail.com] > >>>> >> >> > Sent: Wednesday, February 13, 2013 11:32 AM > >>>> >> >> > To: hdfs-user@hadoop.apache.org > >>>> >> >> > Subject: Managing space in Master Node > >>>> >> >> > > >>>> >> >> > Hello Gurus, > >>>> >> >> > > >>>> >> >> > I am managing a Hadoop Cluster to run some experiments. > >>>> >> >> > > >>>> >> >> > The issue I am continuously facing is that the Master Node > runs > >>>> >> >> > out > >>>> >> >> > of > >>>> >> >> > disk > >>>> >> >> > space due to logs and data files. > >>>> >> >> > > >>>> >> >> > I can monitor and delete log files. However, I cannot delete > the > >>>> >> >> > HDFS > >>>> >> >> > data. > >>>> >> >> > > >>>> >> >> > Thus, is there a way to force Hadoop not to save any HDFS > data in > >>>> >> >> > the > >>>> >> >> > Master > >>>> >> >> > Node? > >>>> >> >> > > >>>> >> >> > Then I can use my master to handle the metadata only and > store the > >>>> >> >> > logs. > >>>> >> >> > > >>>> >> >> > Thanks & regards > >>>> >> >> > Arko > >>>> >> >> > SDL Enterprise Technologies, Inc. - all rights reserved. The > >>>> >> >> > information contained in this email may be confidential and/or > >>>> >> >> > legally > >>>> >> >> > privileged. It has been sent for the sole use of the intended > >>>> >> >> > recipient(s). > >>>> >> >> > If you are not the intended recipient of this mail, you are > hereby > >>>> >> >> > notified > >>>> >> >> > that any unauthorized review, use, disclosure, dissemination, > >>>> >> >> > distribution, > >>>> >> >> > or copying of this communication, or any of its contents, is > >>>> >> >> > strictly > >>>> >> >> > prohibited. If you have received this communication in error, > >>>> >> >> > please > >>>> >> >> > reply > >>>> >> >> > to the sender and destroy all copies of the message. > >>>> >> >> > Registered address: 201 Edgewater Drive, Suite 225, > Wakefield, MA > >>>> >> >> > 01880, > >>>> >> >> > USA > >>>> >> >> > > >>>> >> > > >>>> >> > > >>>> > > >>>> > > >>> > >>> >
