Thanks Bharath and Yang. It is clear now. On Thu, 19 Dec 2019, 04:04 Bharath Kumara Subramanian, < codin.mart...@gmail.com> wrote:
> Hi Debraj, > > I forgot to call this out earlier. Some distribution of YARN doesn't > support node label and rack combination as part of the same request. If you > were to use node labels along with host affinity feature > < > https://samza.apache.org/learn/documentation/latest/yarn/yarn-host-affinity.html > > > in Samza, you might run into following issue > > 19:25:10.032 [main] ClusterBasedJobCoordinator [ERROR] Exception thrown in > > the JobCoordinator loop > > org.apache.hadoop.yarn.client.api.InvalidContainerRequestException: > Cannot > > specify node label with rack and node at > > > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.checkNodeLabelExpression(AMRMClientImpl.java:617) > > at > > > Refer https://jira.apache.org/jira/browse/YARN-4925 > <https://jira.apache.org/jira/browse/YARN-4925?attachmentOrder=asc> for > more information. You may want to back-port the patch to your custom YARN > distribution if applicable. > > Thanks, > Bharath > > On Wed, Dec 18, 2019 at 1:15 PM Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > > Hi Debraj, > > > > To get the node label working, set the label configurations[1] pointed > out > > by Yang in your application config. Samza will take care of embedding the > > node label in the resource request automatically if it notices the label > > configuration inside your application. > > Samza framework respects node label configurations even though they are > > documented in the configuration table. I have created SAMZA-2422 > > <https://issues.apache.org/jira/browse/SAMZA-2422> to track this work > > item. > > > > Let us know if you run into any issues. > > > > Thanks, > > Bharath > > > > [1] - > > *yarn.container.label* for specifying node label for the containers > > *yarn.am.container.label* for specifying node label for the application > > master > > > > On Wed, Dec 18, 2019 at 10:49 AM Debraj Manna <subharaj.ma...@gmail.com> > > wrote: > > > >> I understood how I can assign labels to yarn nodes. > >> > >> But it is still not clear to me how can I specify the node label for a > >> samza application. I am referring to the section "Specifying node label > >> for > >> application" in the link > >> < > >> > https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/NodeLabel.html > >> > > >> you shared in your last email. > >> > >> On Wed, Dec 18, 2019 at 11:17 PM Yang Zhang <zhan3...@umn.edu> wrote: > >> > >> > Hi Debraj Manna, > >> > > >> > The app-def in previous email is just an example where you can > configure > >> > node labels. Yarn node labels > >> > < > >> > > >> > https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/NodeLabel.html > >> > > > >> > is > >> > a general feature (not specific to Samza), and it depends on the > >> > configuration system your system uses. The example uses xml format to > >> > configure Samza job, but Samza as a framework, it does not restrict > >> > configuration format. Please let us know if you have further > questions, > >> and > >> > we should detail the documents in OSS to describe the usage of certain > >> > features. > >> > > >> > Best, > >> > Yang > >> > > >> > On Tue, Dec 17, 2019 at 9:58 PM Debraj Manna < > subharaj.ma...@gmail.com> > >> > wrote: > >> > > >> > > Thanks, Yang for replaying. > >> > > > >> > > Yes, my use case is almost similar. > >> > > > >> > > Can you let me know which app-def you are referring to? I am not > able > >> to > >> > > locate yarn.am.container.label in samza-configurations > >> > > < > >> > > > >> > > >> > http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html > >> > > > > >> > > . > >> > > Is there any samza project whose code I can refer to regarding the > >> usage > >> > of > >> > > these configurations? > >> > > > >> > > On Wed, Dec 18, 2019 at 7:42 AM Yang Zhang <zhan3...@umn.edu> > wrote: > >> > > > >> > > > Hello Debraj, > >> > > > > >> > > > We do not have a formal documentation in open source to describe > how > >> > yarn > >> > > > node label is used in general. In contrast, we have an example of > >> using > >> > > > yarn node label to specify Samza container to run over "HDD" > rather > >> > than > >> > > > default "SSD" nodes. Please take a look at the following guide and > >> let > >> > us > >> > > > know whether it can be applied for your use case. Thank you for > >> > reporting > >> > > > this issue! > >> > > > =================================================Step-by-step > guide > >> > > > > >> > > > > >> > > > 1. > >> > > > > >> > > > Add the *yarn.container.label *and* yarn.am.container.label* to > >> the > >> > > > job's *app-def* if not already present. The default of an empty > >> > string > >> > > > will keep the current default behavior of using SSD nodes. > >> > > > <?xml version="1.0" encoding="UTF-8"?> > >> > > > <application > >> > xmlns="urn:com:linkedin:ns:configuration:definition:1.0" > >> > > > name="my-application" version=""> > >> > > > <configuration-definition> > >> > > > <property name="yarn.am.container.label" default="" /> > >> <!-- > >> > > the > >> > > > label used for launching the application master --> > >> > > > <property name="yarn.container.label" default="" /> > <!-- > >> the > >> > > > label used for other containers --> > >> > > > </configuration-definition> > >> > > > </application> > >> > > > 2. If you had to modify your *app-def* in step 1, you will need > >> to > >> > do > >> > > a > >> > > > trigger-build to get the change to take effect. > >> > > > 3. > >> > > > > >> > > > Add the label to *application.src* for your job. The *hdd* > label > >> > will > >> > > > assign your containers to machines with spinning disks instead > of > >> > > solid > >> > > > state disks. > >> > > > <?xml version="1.0" encoding="UTF-8"?> > >> > > > <application > xmlns="urn:com:linkedin:ns:configuration:source:1.0" > >> > > > name="my-application"> > >> > > > <configuration-source> > >> > > > <property name="yarn.container.label" value="hdd" /> > >> > > > <property name="yarn.am.container.label" value="hdd" /> > >> > > > </configuration-source> > >> > > > </application> > >> > > > > >> > > > > >> > > > 4. > >> > > > > >> > > > Deploy. > >> > > > > >> > > > ================================================= > >> > > > > >> > > > > >> > > > Best, > >> > > > > >> > > > Yang > >> > > > > >> > > > On Tue, Dec 17, 2019 at 10:13 AM Debraj Manna < > >> > subharaj.ma...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > Hi > >> > > > > > >> > > > > I am seeing running samza with yarn node label is resolved in > >> 0.12. > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel > >> > > > > > >> > > > > But I am not able to locate the relevant documentation in > >> samza-yarn > >> > > > > documentation > >> > > > > < > >> > > > > >> > > >> > https://samza.apache.org/learn/documentation/latest/deployment/yarn.html > >> > > > > >> > > > > > >> > > > > Can someone point me to the relevant documentation? > >> > > > > > >> > > > > >> > > > >> > > >> > > >
