John,
The format is ID and sub-config based:
First, you define an ID as a service, like the string "foo". This is
the ID the applications may lookup in their container responses map we
discussed over another thread (around shuffle handler).
<property>
<name>yarn.nodemanager.aux-services</name>
<value>foo</value>
</property>
Then you define an actual implementation class for that ID "foo", like so:
<property>
<name>yarn.nodemanager.aux-services.foo.class</name>
<value>com.mypack.MyAuxServiceClassForFoo</value>
</property>
If you have multiple services foo and bar, then it would appear like
the below (comma separated IDs and individual configs):
<property>
<name>yarn.nodemanager.aux-services</name>
<value>foo,bar</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.foo.class</name>
<value>com.mypack.MyAuxServiceClassForFoo</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.bar.class</name>
<value>com.mypack.MyAuxServiceClassForBar</value>
</property>
On Wed, Jun 5, 2013 at 8:42 PM, John Lilley <[email protected]> wrote:
> Good, I was hoping that would be the case. But what are the mechanics of it?
> Do I just add another entry? And what exactly is "madreduce.shuffle"? A
> scoped class name? Or a key string into some map elsewhere?
>
> e.g. like:
>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>mapreduce.shuffle</value>
> </property>
> <property>
> <name>yarn.nodemanager.aux-services</name>
> <value>myauxserviceclassname</value>
> </property>
>
> Concerning auxiliary services -- do they communicate with NodeManager via
> RPC? Is there an interface to implement? How are they opened and closed
> with NodeManager?
>
> Thanks
> John
>
> -----Original Message-----
> From: Harsh J [mailto:[email protected]]
> Sent: Tuesday, June 04, 2013 11:58 PM
> To: <[email protected]>
> Subject: Re: yarn-site.xml and aux-services
>
> Yes, thats what this is for. You can implement, pass in and use your own
> AuxService. It needs to be on the NodeManager CLASSPATH to run (and NM has to
> be restarted to apply).
>
> On Wed, Jun 5, 2013 at 4:00 AM, John Lilley <[email protected]> wrote:
>> I notice the yarn-site.xml
>>
>>
>>
>> <property>
>>
>> <name>yarn.nodemanager.aux-services</name>
>>
>> <value>mapreduce.shuffle</value>
>>
>> <description>shuffle service that needs to be set for Map Reduce
>> to run </description>
>>
>> </property>
>>
>>
>>
>> Is this a general-purpose hook?
>>
>> Can I tell yarn to run *my* per-node service?
>>
>> Is there some other way (within the recommended Hadoop framework) to
>> run a per-node service that exists during the lifetime of the NodeManager?
>>
>>
>>
>> John Lilley
>>
>> Chief Architect, RedPoint Global Inc.
>>
>> 1515 Walnut Street | Suite 200 | Boulder, CO 80302
>>
>> T: +1 303 541 1516 | M: +1 720 938 5761 | F: +1 781-705-2077
>>
>> Skype: jlilley.redpoint | [email protected] | www.redpoint.net
>>
>>
>
>
>
> --
> Harsh J
--
Harsh J
