If I am not wrong, hadoop.tmp.dir is both local and hdfs. Whatever mount dir, create same in hdfs.
— Sent from Mailbox for iPad On Mon, Dec 16, 2013 at 5:05 PM, Tao Xiao <[email protected]> wrote: > Thanks very much, I suppose I know what I should do with > On Mon, Dec 16, 2013 at 5:27 PM, Vinayakumar B > <[email protected]>wrote: >> Hi, >> >> >> >> *hadoop.tmp.dir* is not the exact configuration you are looking for >> spreading the disk I/O >> >> >> >> This is the default base directory ( its single directory not multiple) >> used in case you didn’t configure your own directories for processes such >> as NameNode, DataNode and NodeManager. >> >> >> >> Exact configurations where you need to configure comma separated values >> are as follows. >> >> *1. **dfs.namenode.name.dir* for namenode in *hdfs-site.xml* >> >> *2. **dfs.datanode.data.dir *for datanode in *hdfs-site.xml* >> >> *3. **yarn.nodemanager.local-dirs* for NodeManager in >> *yarn-site.xml* >> >> >> >> Please note all above configurations are for Hadoop 2.x >> >> >> >> Configure different subdirectories if you are using same disk for multiple >> processes. >> >> Ex: /hadoop/data1/dfs/data >> >> And >> >> /hadoop/data1/yarn/nm-local-dir >> >> >> >> >> >> Cheers, >> >> Vinayakumar B >> >> *From:* Tao Xiao [mailto:[email protected]] >> *Sent:* 16 December 2013 14:42 >> *To:* [email protected] >> *Subject:* Re: How to set "hadoop.tmp.dir" if I have multiple disks per >> node? >> >> >> >> Thanks. >> >> In order to spread I/O among multiple disks, should I assign a >> comma-separated list of directories which are located on different disks to >> "hadoop.tmp.dir"？ >> >> for example, >> >> <property> >> >> <name>hadoop.tmp.dir</name> >> >> >> <value>/mnt/disk1/hadoop_tmp_dir,/mnt/disk2/hadoop_tmp_dir,/mnt/disk3/hadoop_tmp_dir</value> >> >> </property> >> >> >> >> 2013/12/16 Shekhar Sharma <[email protected]> >> >> hadoop.tmp.dir is a directory created on local file system >> For example if you have set hadoop.tmp.dir property to >> /home/training/hadoop >> >> This directory will be created when you format the namenode by running >> the command >> hadoop namenode -format >> >> When you open this folder >> >> >> you will see two subfolders dfs and mapred. >> >> the /home/training/hadoop/mapred folder will be on HDFS also >> >> Hope this clears >> Regards, >> Som Shekhar Sharma >> +91-8197243810 >> >> >> >> On Mon, Dec 16, 2013 at 1:42 PM, Dieter De Witte <[email protected]> >> wrote: >> > Hi, >> > >> > Make sure to also set mapred.local.dir to the same set of output >> > directories, this is were the intermediate key-value pairs are stored! >> > >> > Regards, Dieter >> > >> > >> > 2013/12/16 Tao Xiao <[email protected]> >> >> >> >> I have ten disks per node,and I don't know what value I should set to >> >> "hadoop.tmp.dir". Some said this property refers to a location in local >> disk >> >> while some other said it refers to a directory in HDFS. I'm confused, >> who >> >> can explain it ? >> >> >> >> I want to spread I/O since I have ten disks per node, so should I set a >> >> comma-separated list of directories (which are on different disks) to >> >> "hadoop.tmp.dir" ? >> > >> > >> >> >>
