Thanks! Let me study and find out if it works for me. Thanks & regards Arko
On Wed, Feb 13, 2013 at 2:28 PM, Arko Provo Mukherjee <arkoprovomukher...@gmail.com> wrote: > Hi, > > But the new location will also be the same NN physical server. > So how does that help? > > I would like to clear up the space in the master by removing all the > HDFS data to the slave nodes. > > Thanks & regards > Arko > > On Wed, Feb 13, 2013 at 2:14 PM, Mohammad Tariq <donta...@gmail.com> wrote: >> No, copy this directory to the new location of your choice. >> >> Warm Regards, >> Tariq >> https://mtariq.jux.com/ >> cloudfront.blogspot.com >> >> >> On Thu, Feb 14, 2013 at 1:42 AM, Arko Provo Mukherjee >> <arkoprovomukher...@gmail.com> wrote: >>> >>> Thanks! >>> However, in that case, how do I handle the current data that is already in >>> NN? >>> Do I have to reformat HDFS? >>> Thanks & regards >>> Arko >>> >>> On Wed, Feb 13, 2013 at 2:06 PM, Mohammad Tariq <donta...@gmail.com> >>> wrote: >>> > Just don't run the DN daemon on that machine. Dedicate that machine to >>> > NN. >>> > Remove the hostname of the NN machine from the 'slaves' file. >>> > >>> > Warm Regards, >>> > Tariq >>> > https://mtariq.jux.com/ >>> > cloudfront.blogspot.com >>> > >>> > >>> > On Thu, Feb 14, 2013 at 1:31 AM, Arko Provo Mukherjee >>> > <arkoprovomukher...@gmail.com> wrote: >>> >> >>> >> Hi, >>> >> >>> >> Thanks for the help! >>> >> >>> >> However, I am still unsure about how to "turn off" the datanode >>> >> feature on the NN. >>> >> >>> >> I checked the "hdfs-site.xml" file and the dfs.data.dir is pointed to >>> >> a directory. >>> >> >>> >> Should I just comment out that property? What would happen to the >>> >> current data that there is in the Master? Will it get removed >>> >> automatically? >>> >> >>> >> Thanks & regards >>> >> Arko >>> >> >>> >> >>> >> >>> >> On Wed, Feb 13, 2013 at 1:55 PM, Mohammad Tariq <donta...@gmail.com> >>> >> wrote: >>> >> > You can specify the logging level as specified by Charles. But >>> >> > turning >>> >> > logs >>> >> > off is never a good idea. Logs are really helpful in problem >>> >> > diagnosis, >>> >> > which are eventual. >>> >> > >>> >> > Warm Regards, >>> >> > Tariq >>> >> > https://mtariq.jux.com/ >>> >> > cloudfront.blogspot.com >>> >> > >>> >> > >>> >> > On Thu, Feb 14, 2013 at 1:22 AM, Arko Provo Mukherjee >>> >> > <arkoprovomukher...@gmail.com> wrote: >>> >> >> >>> >> >> Hi, >>> >> >> >>> >> >> Yeah, my NameNode is also seconding as a DataNode. >>> >> >> >>> >> >> I would like to "turn off" this feature. >>> >> >> >>> >> >> Request help regarding the same. >>> >> >> >>> >> >> Thanks & regards >>> >> >> Arko >>> >> >> >>> >> >> On Wed, Feb 13, 2013 at 1:38 PM, Charles Baker <cba...@sdl.com> >>> >> >> wrote: >>> >> >> > Hi Arko. Sounds like you may be running a DataNode on the NameNode >>> >> >> > which >>> >> >> > is >>> >> >> > not recommended practice. Normally, the only files the NN stores >>> >> >> > are >>> >> >> > the >>> >> >> > image and edits files. It does not store any actual HDFS data. If >>> >> >> > you >>> >> >> > must >>> >> >> > run a DN on the NN, try turning down the logging in >>> >> >> > /conf/log4j.properties: >>> >> >> > >>> >> >> > #hadoop.root.logger=INFO,console >>> >> >> > #hadoop.root.logger=WARN,console >>> >> >> > hadoop.root.logger=ERROR,console >>> >> >> > >>> >> >> > Depending on the logging information you require, of course. >>> >> >> > >>> >> >> > -Chuck >>> >> >> > >>> >> >> > >>> >> >> > -----Original Message----- >>> >> >> > From: Arko Provo Mukherjee [mailto:arkoprovomukher...@gmail.com] >>> >> >> > Sent: Wednesday, February 13, 2013 11:32 AM >>> >> >> > To: hdfs-user@hadoop.apache.org >>> >> >> > Subject: Managing space in Master Node >>> >> >> > >>> >> >> > Hello Gurus, >>> >> >> > >>> >> >> > I am managing a Hadoop Cluster to run some experiments. >>> >> >> > >>> >> >> > The issue I am continuously facing is that the Master Node runs >>> >> >> > out >>> >> >> > of >>> >> >> > disk >>> >> >> > space due to logs and data files. >>> >> >> > >>> >> >> > I can monitor and delete log files. However, I cannot delete the >>> >> >> > HDFS >>> >> >> > data. >>> >> >> > >>> >> >> > Thus, is there a way to force Hadoop not to save any HDFS data in >>> >> >> > the >>> >> >> > Master >>> >> >> > Node? >>> >> >> > >>> >> >> > Then I can use my master to handle the metadata only and store the >>> >> >> > logs. >>> >> >> > >>> >> >> > Thanks & regards >>> >> >> > Arko >>> >> >> > SDL Enterprise Technologies, Inc. - all rights reserved. The >>> >> >> > information contained in this email may be confidential and/or >>> >> >> > legally >>> >> >> > privileged. It has been sent for the sole use of the intended >>> >> >> > recipient(s). >>> >> >> > If you are not the intended recipient of this mail, you are hereby >>> >> >> > notified >>> >> >> > that any unauthorized review, use, disclosure, dissemination, >>> >> >> > distribution, >>> >> >> > or copying of this communication, or any of its contents, is >>> >> >> > strictly >>> >> >> > prohibited. If you have received this communication in error, >>> >> >> > please >>> >> >> > reply >>> >> >> > to the sender and destroy all copies of the message. >>> >> >> > Registered address: 201 Edgewater Drive, Suite 225, Wakefield, MA >>> >> >> > 01880, >>> >> >> > USA >>> >> >> > >>> >> > >>> >> > >>> > >>> > >> >>
