RE anirunds question on "how to submit a job remotely". Here are my follow up questions - hope this helps to guide the discussion:
1) Normally - what is the "job client"? Do you guys typically use the namenode as the client? 2) In the case where the client != name node ---- how does the client know how to start up the task trackers ? UCHC On Apr 20, 2012, at 11:19 AM, Amith D K <amit...@huawei.com> wrote: > I dont know your use case if its for test and > ssh across the machine are disabled then u write a script that can do ssh run > the jobs using cli for running your jobs. U can check ssh usage. > > Or else use Ooze > ________________________________________ > From: Robert Evans [ev...@yahoo-inc.com] > Sent: Friday, April 20, 2012 11:17 PM > To: common-user@hadoop.apache.org > Subject: Re: remote job submission > > You can use Oozie to do it. > > > On 4/20/12 8:45 AM, "Arindam Choudhury" <arindamchoudhu...@gmail.com> wrote: > > Sorry. But I can you give me a example. > > On Fri, Apr 20, 2012 at 3:08 PM, Harsh J <ha...@cloudera.com> wrote: > >> Arindam, >> >> If your machine can access the clusters' NN/JT/DN ports, then you can >> simply run your job from the machine itself. >> >> On Fri, Apr 20, 2012 at 6:31 PM, Arindam Choudhury >> <arindamchoudhu...@gmail.com> wrote: >>> "If you are allowed a remote connection to the cluster's service ports, >>> then you can directly submit your jobs from your local CLI. Just make >>> sure your local configuration points to the right locations." >>> >>> Can you elaborate in details please? >>> >>> On Fri, Apr 20, 2012 at 2:20 PM, Harsh J <ha...@cloudera.com> wrote: >>> >>>> If you are allowed a remote connection to the cluster's service ports, >>>> then you can directly submit your jobs from your local CLI. Just make >>>> sure your local configuration points to the right locations. >>>> >>>> Otherwise, perhaps you can choose to use Apache Oozie (Incubating) >>>> (http://incubator.apache.org/oozie/) It does provide a REST interface >>>> that launches jobs up for you over the supplied clusters, but its more >>>> oriented towards workflow management or perhaps HUE: >>>> https://github.com/cloudera/hue >>>> >>>> On Fri, Apr 20, 2012 at 5:37 PM, Arindam Choudhury >>>> <arindamchoudhu...@gmail.com> wrote: >>>>> Hi, >>>>> >>>>> Do hadoop have any web service or other interface so I can submit jobs >>>> from >>>>> remote machine? >>>>> >>>>> Thanks, >>>>> Arindam >>>> >>>> >>>> >>>> -- >>>> Harsh J >>>> >> >> >> >> -- >> Harsh J >> >
