I tried this. So before running the WordCount sample I did an export HADOOP_CONF_DIR=<my conf dir>. Doesn't seem to help. I still see the LocalJobRunner being used.
Thanks Avinash On 5/24/07, Mahadev Konar <[EMAIL PROTECTED]> wrote:
Hi, When you run the job, you need to set the environment variable HADOOP_CONF_DIR to the configuration directory that has the configuration file pointing to the right jobtracker. Regards Mahadev > -----Original Message----- > From: Phantom [mailto:[EMAIL PROTECTED] > Sent: Thursday, May 24, 2007 4:51 PM > To: [email protected] > Subject: Re: Configuration and Hadoop cluster setup > > Yes the files are the same and I am starting the tasks on the namenode > server. I also figured what my problem was with respect to not being able > to > start the namenode and job tracker on the same machine. I had to reformat > the file system. But the all this still doesn't cause the WordCount sample > to run in a distributed fashion. I can tell this because the > LocalJobRunner > is being used. Do I need to specify the config file to the running > instance > of the program ? If so how do I do that ? > > Thanks > A > > On 5/24/07, Dennis Kubes <[EMAIL PROTECTED]> wrote: > > > > > > > > Phantom wrote: > > > I am trying to run Hadoop on a cluster of 3 nodes. The namenode and > the > > > jobtracker web UI work. I have the namenode running on node A and job > > > tracker running on node B. Is it true that namenode and jobtracker > > cannot > > > run on the same box ? > > > > The namenode and the jobtracker can most definitely run on the same box. > > As far as I know this is the preferred configuration. > > > > Also if I want to run the examples on the cluster is > > > there anything special that needs to be done. When I run the example > > > WordCount on a machine C (which is a task tracker and not a job > tracker) > > > the > > > LocalJobRunner is invoked all the time. I am guessing this means that > > the > > > map tasks are running locally. How can I distribute this on the > cluster > > ? > > > Please advice. > > > > Are the conf files on machine C the same as the namenode/jobtracker? > > Are they pointing to the namenode and jobtracker or are they pointing to > > local in the hadoop-site.xml file. Also we have found it easier > > (although not necessarily better) to start tasks on the namenode server. > > > > It would be helpful to have more information about what is happening and > > your setup as that would help myself and others on the list debug what > > may be occurring. > > > > Dennis Kubes > > > > > > > > Thanks > > > Avinash > > > > >
